2023-03-10 18:53:28,922 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-10 18:53:29,208 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c 2023-03-10 18:53:29,223 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-10 18:53:29,224 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/cluster_7b2047bb-341b-b20b-9460-0c7ebaa01809, deleteOnExit=true 2023-03-10 18:53:29,224 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-10 18:53:29,225 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/test.cache.data in system properties and HBase conf 2023-03-10 18:53:29,226 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/hadoop.tmp.dir in system properties and HBase conf 2023-03-10 18:53:29,226 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/hadoop.log.dir in system properties and HBase conf 2023-03-10 18:53:29,227 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-10 18:53:29,227 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-10 18:53:29,227 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-10 18:53:29,349 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-10 18:53:29,764 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-10 18:53:29,769 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-10 18:53:29,770 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-10 18:53:29,770 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-10 18:53:29,771 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-10 18:53:29,771 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-10 18:53:29,772 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-10 18:53:29,772 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-10 18:53:29,773 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-10 18:53:29,773 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-10 18:53:29,773 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/nfs.dump.dir in system properties and HBase conf 2023-03-10 18:53:29,774 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/java.io.tmpdir in system properties and HBase conf 2023-03-10 18:53:29,774 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-10 18:53:29,775 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-10 18:53:29,775 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-10 18:53:30,386 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-10 18:53:30,390 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-10 18:53:30,683 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-10 18:53:30,864 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-10 18:53:30,884 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-10 18:53:30,923 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-10 18:53:30,959 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/java.io.tmpdir/Jetty_localhost_33941_hdfs____tktcqo/webapp 2023-03-10 18:53:31,096 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33941 2023-03-10 18:53:31,132 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-10 18:53:31,133 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-10 18:53:31,592 WARN [Listener at localhost/38247] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-10 18:53:31,721 WARN [Listener at localhost/38247] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-10 18:53:31,743 WARN [Listener at localhost/38247] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-10 18:53:31,751 INFO [Listener at localhost/38247] log.Slf4jLog(67): jetty-6.1.26 2023-03-10 18:53:31,758 INFO [Listener at localhost/38247] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/java.io.tmpdir/Jetty_localhost_40401_datanode____.y1nj34/webapp 2023-03-10 18:53:31,867 INFO [Listener at localhost/38247] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40401 2023-03-10 18:53:32,233 WARN [Listener at localhost/36413] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-10 18:53:32,766 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe1692531617b7e54: Processing first storage report for DS-9d5a37d8-4d1b-48cf-9204-03faffbcbdcc from datanode 9b110239-679c-48ee-87dd-e592745d4fb2 2023-03-10 18:53:32,768 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe1692531617b7e54: from storage DS-9d5a37d8-4d1b-48cf-9204-03faffbcbdcc node DatanodeRegistration(127.0.0.1:40241, datanodeUuid=9b110239-679c-48ee-87dd-e592745d4fb2, infoPort=45309, infoSecurePort=0, ipcPort=36413, storageInfo=lv=-57;cid=testClusterID;nsid=1009369534;c=1678474410466), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-03-10 18:53:32,769 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe1692531617b7e54: Processing first storage report for DS-18afcd66-a851-4470-8a2b-3234f66400cc from datanode 9b110239-679c-48ee-87dd-e592745d4fb2 2023-03-10 18:53:32,769 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe1692531617b7e54: from storage DS-18afcd66-a851-4470-8a2b-3234f66400cc node DatanodeRegistration(127.0.0.1:40241, datanodeUuid=9b110239-679c-48ee-87dd-e592745d4fb2, infoPort=45309, infoSecurePort=0, ipcPort=36413, storageInfo=lv=-57;cid=testClusterID;nsid=1009369534;c=1678474410466), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-10 18:53:32,847 DEBUG [Listener at localhost/36413] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c 2023-03-10 18:53:32,912 INFO [Listener at localhost/36413] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/cluster_7b2047bb-341b-b20b-9460-0c7ebaa01809/zookeeper_0, clientPort=56431, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/cluster_7b2047bb-341b-b20b-9460-0c7ebaa01809/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/cluster_7b2047bb-341b-b20b-9460-0c7ebaa01809/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-10 18:53:32,925 INFO [Listener at localhost/36413] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=56431 2023-03-10 18:53:32,932 INFO [Listener at localhost/36413] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-10 18:53:32,935 INFO [Listener at localhost/36413] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-10 18:53:33,588 INFO [Listener at localhost/36413] util.FSUtils(471): Created version file at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5 with version=8 2023-03-10 18:53:33,588 INFO [Listener at localhost/36413] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/hbase-staging 2023-03-10 18:53:33,907 INFO [Listener at localhost/36413] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-10 18:53:34,359 INFO [Listener at localhost/36413] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-03-10 18:53:34,390 INFO [Listener at localhost/36413] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-10 18:53:34,391 INFO [Listener at localhost/36413] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-10 18:53:34,391 INFO [Listener at localhost/36413] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-10 18:53:34,391 INFO [Listener at localhost/36413] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-10 18:53:34,392 INFO [Listener at localhost/36413] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-10 18:53:34,577 INFO [Listener at localhost/36413] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-10 18:53:34,648 DEBUG [Listener at localhost/36413] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-10 18:53:34,745 INFO [Listener at localhost/36413] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33867 2023-03-10 18:53:34,760 INFO [Listener at localhost/36413] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-10 18:53:34,763 INFO [Listener at localhost/36413] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-10 18:53:34,791 INFO [Listener at localhost/36413] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33867 connecting to ZooKeeper ensemble=127.0.0.1:56431 2023-03-10 18:53:34,833 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:338670x0, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-10 18:53:34,836 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:33867-0x100cfc6ed280000 connected 2023-03-10 18:53:34,868 DEBUG [Listener at localhost/36413] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-10 18:53:34,869 DEBUG [Listener at localhost/36413] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-10 18:53:34,872 DEBUG [Listener at localhost/36413] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-10 18:53:34,880 DEBUG [Listener at localhost/36413] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33867 2023-03-10 18:53:34,881 DEBUG [Listener at localhost/36413] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33867 2023-03-10 18:53:34,881 DEBUG [Listener at localhost/36413] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33867 2023-03-10 18:53:34,881 DEBUG [Listener at localhost/36413] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33867 2023-03-10 18:53:34,884 DEBUG [Listener at localhost/36413] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33867 2023-03-10 18:53:34,890 INFO [Listener at localhost/36413] master.HMaster(444): hbase.rootdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5, hbase.cluster.distributed=false 2023-03-10 18:53:34,960 INFO [Listener at localhost/36413] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-03-10 18:53:34,961 INFO [Listener at localhost/36413] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-10 18:53:34,961 INFO [Listener at localhost/36413] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-10 18:53:34,961 INFO [Listener at localhost/36413] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-10 18:53:34,961 INFO [Listener at localhost/36413] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-10 18:53:34,961 INFO [Listener at localhost/36413] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-10 18:53:34,966 INFO [Listener at localhost/36413] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-10 18:53:34,969 INFO [Listener at localhost/36413] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43181 2023-03-10 18:53:34,972 INFO [Listener at localhost/36413] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-10 18:53:34,978 DEBUG [Listener at localhost/36413] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-10 18:53:34,979 INFO [Listener at localhost/36413] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-10 18:53:34,983 INFO [Listener at localhost/36413] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-10 18:53:34,984 INFO [Listener at localhost/36413] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43181 connecting to ZooKeeper ensemble=127.0.0.1:56431 2023-03-10 18:53:34,988 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:431810x0, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-10 18:53:34,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43181-0x100cfc6ed280001 connected 2023-03-10 18:53:34,989 DEBUG [Listener at localhost/36413] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-10 18:53:34,991 DEBUG [Listener at localhost/36413] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-10 18:53:34,992 DEBUG [Listener at localhost/36413] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-10 18:53:34,992 DEBUG [Listener at localhost/36413] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43181 2023-03-10 18:53:34,993 DEBUG [Listener at localhost/36413] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43181 2023-03-10 18:53:34,996 DEBUG [Listener at localhost/36413] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43181 2023-03-10 18:53:34,997 DEBUG [Listener at localhost/36413] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43181 2023-03-10 18:53:34,997 DEBUG [Listener at localhost/36413] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43181 2023-03-10 18:53:34,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33867,1678474413725 2023-03-10 18:53:35,009 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-10 18:53:35,011 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33867,1678474413725 2023-03-10 18:53:35,031 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-10 18:53:35,031 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-10 18:53:35,032 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-10 18:53:35,033 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-10 18:53:35,034 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33867,1678474413725 from backup master directory 2023-03-10 18:53:35,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-10 18:53:35,037 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33867,1678474413725 2023-03-10 18:53:35,037 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-10 18:53:35,038 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-10 18:53:35,038 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33867,1678474413725 2023-03-10 18:53:35,041 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-10 18:53:35,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-10 18:53:35,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/hbase.id with ID: 7000002c-ea3f-4400-95f4-b579e870154c 2023-03-10 18:53:35,174 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-10 18:53:35,193 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-10 18:53:35,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4d3c7d3c to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:53:35,268 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@901755e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:53:35,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-10 18:53:35,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-10 18:53:35,315 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-10 18:53:35,315 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-10 18:53:35,318 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-10 18:53:35,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-10 18:53:35,324 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-10 18:53:35,362 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/data/master/store-tmp 2023-03-10 18:53:35,791 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:53:35,791 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-10 18:53:35,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-10 18:53:35,792 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-10 18:53:35,792 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-10 18:53:35,792 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-10 18:53:35,792 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-10 18:53:35,792 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-10 18:53:35,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/WALs/jenkins-hbase4.apache.org,33867,1678474413725 2023-03-10 18:53:35,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33867%2C1678474413725, suffix=, logDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/WALs/jenkins-hbase4.apache.org,33867,1678474413725, archiveDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/oldWALs, maxLogs=10 2023-03-10 18:53:35,883 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40241,DS-9d5a37d8-4d1b-48cf-9204-03faffbcbdcc,DISK] 2023-03-10 18:53:35,891 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-10 18:53:35,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/WALs/jenkins-hbase4.apache.org,33867,1678474413725/jenkins-hbase4.apache.org%2C33867%2C1678474413725.1678474415825 2023-03-10 18:53:35,964 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40241,DS-9d5a37d8-4d1b-48cf-9204-03faffbcbdcc,DISK]] 2023-03-10 18:53:35,965 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:53:35,965 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:53:35,969 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-10 18:53:35,971 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-10 18:53:36,038 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-10 18:53:36,045 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-10 18:53:36,074 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-10 18:53:36,088 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:53:36,093 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-10 18:53:36,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-10 18:53:36,111 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-10 18:53:36,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:53:36,117 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59064977, jitterRate=-0.11986325681209564}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:53:36,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-10 18:53:36,119 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-10 18:53:36,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-10 18:53:36,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-10 18:53:36,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-10 18:53:36,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-10 18:53:36,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 37 msec 2023-03-10 18:53:36,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-10 18:53:36,218 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-10 18:53:36,224 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-10 18:53:36,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-10 18:53:36,251 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-10 18:53:36,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-10 18:53:36,257 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-10 18:53:36,261 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-10 18:53:36,265 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-10 18:53:36,266 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-10 18:53:36,266 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-10 18:53:36,280 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-10 18:53:36,286 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-10 18:53:36,286 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-10 18:53:36,286 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-10 18:53:36,287 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33867,1678474413725, sessionid=0x100cfc6ed280000, setting cluster-up flag (Was=false) 2023-03-10 18:53:36,308 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-10 18:53:36,317 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-10 18:53:36,318 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33867,1678474413725 2023-03-10 18:53:36,324 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-10 18:53:36,330 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-10 18:53:36,332 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33867,1678474413725 2023-03-10 18:53:36,335 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.hbase-snapshot/.tmp 2023-03-10 18:53:36,402 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(952): ClusterId : 7000002c-ea3f-4400-95f4-b579e870154c 2023-03-10 18:53:36,408 DEBUG [RS:0;jenkins-hbase4:43181] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-10 18:53:36,416 DEBUG [RS:0;jenkins-hbase4:43181] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-10 18:53:36,416 DEBUG [RS:0;jenkins-hbase4:43181] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-10 18:53:36,426 DEBUG [RS:0;jenkins-hbase4:43181] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-10 18:53:36,428 DEBUG [RS:0;jenkins-hbase4:43181] zookeeper.ReadOnlyZKClient(139): Connect 0x23e363a3 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:53:36,452 DEBUG [RS:0;jenkins-hbase4:43181] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d2499af, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:53:36,452 DEBUG [RS:0;jenkins-hbase4:43181] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@291b08d3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-10 18:53:36,485 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:43181 2023-03-10 18:53:36,488 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-10 18:53:36,488 INFO [RS:0;jenkins-hbase4:43181] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-10 18:53:36,489 INFO [RS:0;jenkins-hbase4:43181] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-10 18:53:36,489 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1023): About to register with Master. 2023-03-10 18:53:36,492 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33867,1678474413725 with isa=jenkins-hbase4.apache.org/172.31.14.131:43181, startcode=1678474414959 2023-03-10 18:53:36,500 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-10 18:53:36,500 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-10 18:53:36,500 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-10 18:53:36,501 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-10 18:53:36,501 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-03-10 18:53:36,501 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-10 18:53:36,501 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-10 18:53:36,501 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-10 18:53:36,504 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1678474446504 2023-03-10 18:53:36,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-10 18:53:36,509 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-10 18:53:36,510 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-10 18:53:36,512 DEBUG [RS:0;jenkins-hbase4:43181] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-10 18:53:36,515 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-10 18:53:36,519 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-10 18:53:36,528 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-10 18:53:36,528 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-10 18:53:36,529 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-10 18:53:36,529 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-10 18:53:36,529 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:36,536 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-10 18:53:36,538 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-10 18:53:36,538 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-10 18:53:36,549 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-10 18:53:36,549 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-10 18:53:36,560 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678474416556,5,FailOnTimeoutGroup] 2023-03-10 18:53:36,566 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678474416565,5,FailOnTimeoutGroup] 2023-03-10 18:53:36,566 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:36,566 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-10 18:53:36,570 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:36,570 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:36,590 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54133, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-10 18:53:36,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:36,622 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5 2023-03-10 18:53:36,622 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38247 2023-03-10 18:53:36,622 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-10 18:53:36,628 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-10 18:53:36,629 DEBUG [RS:0;jenkins-hbase4:43181] zookeeper.ZKUtil(162): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:36,630 WARN [RS:0;jenkins-hbase4:43181] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-10 18:53:36,630 INFO [RS:0;jenkins-hbase4:43181] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-10 18:53:36,633 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/WALs/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:36,631 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43181,1678474414959] 2023-03-10 18:53:36,652 DEBUG [RS:0;jenkins-hbase4:43181] zookeeper.ZKUtil(162): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:36,664 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-10 18:53:36,674 INFO [RS:0;jenkins-hbase4:43181] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-10 18:53:36,698 INFO [RS:0;jenkins-hbase4:43181] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-10 18:53:36,702 INFO [RS:0;jenkins-hbase4:43181] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:53:36,702 INFO [RS:0;jenkins-hbase4:43181] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:36,703 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-10 18:53:36,712 INFO [RS:0;jenkins-hbase4:43181] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-10 18:53:36,712 DEBUG [RS:0;jenkins-hbase4:43181] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-10 18:53:36,713 DEBUG [RS:0;jenkins-hbase4:43181] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-10 18:53:36,713 DEBUG [RS:0;jenkins-hbase4:43181] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-10 18:53:36,713 DEBUG [RS:0;jenkins-hbase4:43181] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-10 18:53:36,714 DEBUG [RS:0;jenkins-hbase4:43181] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-10 18:53:36,714 DEBUG [RS:0;jenkins-hbase4:43181] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-10 18:53:36,714 DEBUG [RS:0;jenkins-hbase4:43181] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-10 18:53:36,714 DEBUG [RS:0;jenkins-hbase4:43181] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-10 18:53:36,714 DEBUG [RS:0;jenkins-hbase4:43181] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-10 18:53:36,714 DEBUG [RS:0;jenkins-hbase4:43181] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-10 18:53:36,720 INFO [RS:0;jenkins-hbase4:43181] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:36,721 INFO [RS:0;jenkins-hbase4:43181] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:36,721 INFO [RS:0;jenkins-hbase4:43181] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:36,739 INFO [RS:0;jenkins-hbase4:43181] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-10 18:53:36,742 INFO [RS:0;jenkins-hbase4:43181] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43181,1678474414959-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:36,758 INFO [RS:0;jenkins-hbase4:43181] regionserver.Replication(203): jenkins-hbase4.apache.org,43181,1678474414959 started 2023-03-10 18:53:36,758 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,43181,1678474414959, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43181, sessionid=0x100cfc6ed280001 2023-03-10 18:53:36,759 DEBUG [RS:0;jenkins-hbase4:43181] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-10 18:53:36,759 DEBUG [RS:0;jenkins-hbase4:43181] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:36,759 DEBUG [RS:0;jenkins-hbase4:43181] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43181,1678474414959' 2023-03-10 18:53:36,759 DEBUG [RS:0;jenkins-hbase4:43181] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:36,760 DEBUG [RS:0;jenkins-hbase4:43181] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:36,760 DEBUG [RS:0;jenkins-hbase4:43181] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-10 18:53:36,760 DEBUG [RS:0;jenkins-hbase4:43181] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-10 18:53:36,760 DEBUG [RS:0;jenkins-hbase4:43181] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:36,760 DEBUG [RS:0;jenkins-hbase4:43181] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43181,1678474414959' 2023-03-10 18:53:36,760 DEBUG [RS:0;jenkins-hbase4:43181] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-10 18:53:36,761 DEBUG [RS:0;jenkins-hbase4:43181] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-10 18:53:36,761 DEBUG [RS:0;jenkins-hbase4:43181] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-10 18:53:36,761 INFO [RS:0;jenkins-hbase4:43181] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-10 18:53:36,761 INFO [RS:0;jenkins-hbase4:43181] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-10 18:53:36,874 INFO [RS:0;jenkins-hbase4:43181] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43181%2C1678474414959, suffix=, logDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/WALs/jenkins-hbase4.apache.org,43181,1678474414959, archiveDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/oldWALs, maxLogs=32 2023-03-10 18:53:36,893 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40241,DS-9d5a37d8-4d1b-48cf-9204-03faffbcbdcc,DISK] 2023-03-10 18:53:36,897 INFO [RS:0;jenkins-hbase4:43181] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/WALs/jenkins-hbase4.apache.org,43181,1678474414959/jenkins-hbase4.apache.org%2C43181%2C1678474414959.1678474416877 2023-03-10 18:53:36,898 DEBUG [RS:0;jenkins-hbase4:43181] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40241,DS-9d5a37d8-4d1b-48cf-9204-03faffbcbdcc,DISK]] 2023-03-10 18:53:36,976 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-10 18:53:36,977 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-10 18:53:36,977 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5 2023-03-10 18:53:37,391 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:53:37,394 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-10 18:53:37,398 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/info 2023-03-10 18:53:37,399 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-10 18:53:37,401 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:53:37,401 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-10 18:53:37,404 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/rep_barrier 2023-03-10 18:53:37,404 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-10 18:53:37,405 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:53:37,405 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-10 18:53:37,408 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/table 2023-03-10 18:53:37,408 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-10 18:53:37,409 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:53:37,411 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740 2023-03-10 18:53:37,412 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740 2023-03-10 18:53:37,416 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-10 18:53:37,418 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-10 18:53:37,422 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:53:37,423 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68021863, jitterRate=0.013604745268821716}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-10 18:53:37,423 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-10 18:53:37,423 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-10 18:53:37,423 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-10 18:53:37,423 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-10 18:53:37,423 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-10 18:53:37,423 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-10 18:53:37,424 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-10 18:53:37,425 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-10 18:53:37,430 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-10 18:53:37,430 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-10 18:53:37,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-10 18:53:37,452 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-10 18:53:37,455 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-10 18:53:37,606 DEBUG [jenkins-hbase4:33867] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-10 18:53:37,610 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43181,1678474414959, state=OPENING 2023-03-10 18:53:37,617 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-10 18:53:37,619 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-10 18:53:37,619 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-10 18:53:37,621 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:53:37,804 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:37,806 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-10 18:53:37,810 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52420, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-10 18:53:37,820 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-10 18:53:37,820 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-10 18:53:37,824 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43181%2C1678474414959.meta, suffix=.meta, logDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/WALs/jenkins-hbase4.apache.org,43181,1678474414959, archiveDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/oldWALs, maxLogs=32 2023-03-10 18:53:37,842 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40241,DS-9d5a37d8-4d1b-48cf-9204-03faffbcbdcc,DISK] 2023-03-10 18:53:37,847 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/WALs/jenkins-hbase4.apache.org,43181,1678474414959/jenkins-hbase4.apache.org%2C43181%2C1678474414959.meta.1678474417826.meta 2023-03-10 18:53:37,847 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40241,DS-9d5a37d8-4d1b-48cf-9204-03faffbcbdcc,DISK]] 2023-03-10 18:53:37,847 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:53:37,849 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-10 18:53:37,869 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-10 18:53:37,875 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-10 18:53:37,881 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-10 18:53:37,881 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:53:37,881 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-10 18:53:37,882 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-10 18:53:37,886 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-10 18:53:37,888 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/info 2023-03-10 18:53:37,888 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/info 2023-03-10 18:53:37,889 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-10 18:53:37,890 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:53:37,890 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-10 18:53:37,891 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/rep_barrier 2023-03-10 18:53:37,892 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/rep_barrier 2023-03-10 18:53:37,892 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-10 18:53:37,893 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:53:37,893 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-10 18:53:37,895 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/table 2023-03-10 18:53:37,895 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/table 2023-03-10 18:53:37,896 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-10 18:53:37,897 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:53:37,899 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740 2023-03-10 18:53:37,902 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740 2023-03-10 18:53:37,906 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-10 18:53:37,909 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-10 18:53:37,911 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64268600, jitterRate=-0.04232323169708252}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-10 18:53:37,911 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-10 18:53:37,920 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1678474417796 2023-03-10 18:53:37,941 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-10 18:53:37,941 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-10 18:53:37,942 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43181,1678474414959, state=OPEN 2023-03-10 18:53:37,947 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-10 18:53:37,947 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-10 18:53:37,956 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-10 18:53:37,956 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43181,1678474414959 in 326 msec 2023-03-10 18:53:37,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-10 18:53:37,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 517 msec 2023-03-10 18:53:37,967 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5730 sec 2023-03-10 18:53:37,967 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1678474417967, completionTime=-1 2023-03-10 18:53:37,968 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-10 18:53:37,968 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-03-10 18:53:38,034 DEBUG [hconnection-0x4fd27954-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:53:38,038 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52434, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:53:38,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-03-10 18:53:38,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1678474478059 2023-03-10 18:53:38,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1678474538059 2023-03-10 18:53:38,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 90 msec 2023-03-10 18:53:38,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33867,1678474413725-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:38,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33867,1678474413725-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:38,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33867,1678474413725-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:38,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33867, period=300000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:38,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-10 18:53:38,100 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-10 18:53:38,108 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-10 18:53:38,110 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-10 18:53:38,119 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-10 18:53:38,122 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:53:38,125 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:53:38,145 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679 2023-03-10 18:53:38,148 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679 empty. 2023-03-10 18:53:38,148 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679 2023-03-10 18:53:38,148 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-10 18:53:38,572 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-10 18:53:38,574 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6b12c67bd3abfa00d30f0e4b35405679, NAME => 'hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:53:38,588 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:53:38,588 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 6b12c67bd3abfa00d30f0e4b35405679, disabling compactions & flushes 2023-03-10 18:53:38,588 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:53:38,588 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:53:38,588 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. after waiting 0 ms 2023-03-10 18:53:38,588 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:53:38,588 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:53:38,588 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 6b12c67bd3abfa00d30f0e4b35405679: 2023-03-10 18:53:38,593 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:53:38,609 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678474418596"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474418596"}]},"ts":"1678474418596"} 2023-03-10 18:53:38,635 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-10 18:53:38,637 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:53:38,642 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474418638"}]},"ts":"1678474418638"} 2023-03-10 18:53:38,646 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-10 18:53:38,658 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6b12c67bd3abfa00d30f0e4b35405679, ASSIGN}] 2023-03-10 18:53:38,660 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6b12c67bd3abfa00d30f0e4b35405679, ASSIGN 2023-03-10 18:53:38,662 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=6b12c67bd3abfa00d30f0e4b35405679, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:53:38,813 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6b12c67bd3abfa00d30f0e4b35405679, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:38,813 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678474418812"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474418812"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474418812"}]},"ts":"1678474418812"} 2023-03-10 18:53:38,817 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 6b12c67bd3abfa00d30f0e4b35405679, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:53:38,979 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:53:38,979 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6b12c67bd3abfa00d30f0e4b35405679, NAME => 'hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:53:38,981 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 6b12c67bd3abfa00d30f0e4b35405679 2023-03-10 18:53:38,981 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:53:38,981 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6b12c67bd3abfa00d30f0e4b35405679 2023-03-10 18:53:38,981 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6b12c67bd3abfa00d30f0e4b35405679 2023-03-10 18:53:38,984 INFO [StoreOpener-6b12c67bd3abfa00d30f0e4b35405679-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 6b12c67bd3abfa00d30f0e4b35405679 2023-03-10 18:53:38,986 DEBUG [StoreOpener-6b12c67bd3abfa00d30f0e4b35405679-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679/info 2023-03-10 18:53:38,986 DEBUG [StoreOpener-6b12c67bd3abfa00d30f0e4b35405679-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679/info 2023-03-10 18:53:38,987 INFO [StoreOpener-6b12c67bd3abfa00d30f0e4b35405679-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b12c67bd3abfa00d30f0e4b35405679 columnFamilyName info 2023-03-10 18:53:38,987 INFO [StoreOpener-6b12c67bd3abfa00d30f0e4b35405679-1] regionserver.HStore(310): Store=6b12c67bd3abfa00d30f0e4b35405679/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:53:38,989 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679 2023-03-10 18:53:38,990 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679 2023-03-10 18:53:38,996 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6b12c67bd3abfa00d30f0e4b35405679 2023-03-10 18:53:39,001 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:53:39,002 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6b12c67bd3abfa00d30f0e4b35405679; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64878136, jitterRate=-0.033240437507629395}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:53:39,003 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6b12c67bd3abfa00d30f0e4b35405679: 2023-03-10 18:53:39,005 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679., pid=6, masterSystemTime=1678474418972 2023-03-10 18:53:39,009 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:53:39,009 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:53:39,011 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6b12c67bd3abfa00d30f0e4b35405679, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:39,011 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678474419010"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474419010"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474419010"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474419010"}]},"ts":"1678474419010"} 2023-03-10 18:53:39,020 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-10 18:53:39,020 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 6b12c67bd3abfa00d30f0e4b35405679, server=jenkins-hbase4.apache.org,43181,1678474414959 in 199 msec 2023-03-10 18:53:39,025 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-10 18:53:39,025 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=6b12c67bd3abfa00d30f0e4b35405679, ASSIGN in 362 msec 2023-03-10 18:53:39,027 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:53:39,027 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474419027"}]},"ts":"1678474419027"} 2023-03-10 18:53:39,030 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-10 18:53:39,033 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:53:39,036 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 922 msec 2023-03-10 18:53:39,122 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-10 18:53:39,124 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-10 18:53:39,124 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-10 18:53:39,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-10 18:53:39,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-10 18:53:39,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 38 msec 2023-03-10 18:53:39,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-10 18:53:39,205 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-10 18:53:39,210 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-03-10 18:53:39,221 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-10 18:53:39,223 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-10 18:53:39,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 4.187sec 2023-03-10 18:53:39,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-10 18:53:39,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-10 18:53:39,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-10 18:53:39,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33867,1678474413725-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-10 18:53:39,231 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33867,1678474413725-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-10 18:53:39,241 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-10 18:53:39,308 DEBUG [Listener at localhost/36413] zookeeper.ReadOnlyZKClient(139): Connect 0x18f0808a to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:53:39,314 DEBUG [Listener at localhost/36413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5759ba4d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:53:39,326 DEBUG [hconnection-0x1911755b-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:53:39,341 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52442, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:53:39,352 INFO [Listener at localhost/36413] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,33867,1678474413725 2023-03-10 18:53:39,353 DEBUG [Listener at localhost/36413] zookeeper.ReadOnlyZKClient(139): Connect 0x69d78ff9 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:53:39,358 DEBUG [Listener at localhost/36413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47652242, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:53:39,359 INFO [Listener at localhost/36413] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-10 18:53:39,385 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=313, ProcessCount=170, AvailableMemoryMB=5974 2023-03-10 18:53:39,397 DEBUG [Listener at localhost/36413] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:53:39,400 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53210, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:53:39,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-10 18:53:39,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:39,417 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:53:39,419 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:53:39,421 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-10 18:53:39,421 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:39,423 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c empty. 2023-03-10 18:53:39,424 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:39,424 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-10 18:53:39,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-10 18:53:39,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-10 18:53:39,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-10 18:53:39,842 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-10 18:53:39,844 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => a8073efb693951bd51420aa7456ef98c, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:53:39,856 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:53:39,856 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing a8073efb693951bd51420aa7456ef98c, disabling compactions & flushes 2023-03-10 18:53:39,856 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:39,856 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:39,856 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. after waiting 0 ms 2023-03-10 18:53:39,856 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:39,856 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:39,857 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:39,861 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:53:39,862 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678474419862"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474419862"}]},"ts":"1678474419862"} 2023-03-10 18:53:39,865 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-10 18:53:39,866 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:53:39,866 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474419866"}]},"ts":"1678474419866"} 2023-03-10 18:53:39,869 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-10 18:53:39,873 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a8073efb693951bd51420aa7456ef98c, ASSIGN}] 2023-03-10 18:53:39,875 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a8073efb693951bd51420aa7456ef98c, ASSIGN 2023-03-10 18:53:39,877 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a8073efb693951bd51420aa7456ef98c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:53:40,028 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a8073efb693951bd51420aa7456ef98c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:40,028 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678474420028"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474420028"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474420028"}]},"ts":"1678474420028"} 2023-03-10 18:53:40,031 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure a8073efb693951bd51420aa7456ef98c, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:53:40,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-10 18:53:40,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:40,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a8073efb693951bd51420aa7456ef98c, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:53:40,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:40,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:53:40,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:40,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:40,196 INFO [StoreOpener-a8073efb693951bd51420aa7456ef98c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:40,198 DEBUG [StoreOpener-a8073efb693951bd51420aa7456ef98c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1 2023-03-10 18:53:40,198 DEBUG [StoreOpener-a8073efb693951bd51420aa7456ef98c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1 2023-03-10 18:53:40,202 INFO [StoreOpener-a8073efb693951bd51420aa7456ef98c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a8073efb693951bd51420aa7456ef98c columnFamilyName family1 2023-03-10 18:53:40,205 INFO [StoreOpener-a8073efb693951bd51420aa7456ef98c-1] regionserver.HStore(310): Store=a8073efb693951bd51420aa7456ef98c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:53:40,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:40,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:40,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:40,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:53:40,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a8073efb693951bd51420aa7456ef98c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73838688, jitterRate=0.10028219223022461}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:53:40,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:40,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c., pid=11, masterSystemTime=1678474420185 2023-03-10 18:53:40,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:40,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:40,239 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a8073efb693951bd51420aa7456ef98c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:40,239 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678474420239"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474420239"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474420239"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474420239"}]},"ts":"1678474420239"} 2023-03-10 18:53:40,245 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-10 18:53:40,246 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure a8073efb693951bd51420aa7456ef98c, server=jenkins-hbase4.apache.org,43181,1678474414959 in 211 msec 2023-03-10 18:53:40,249 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-10 18:53:40,249 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a8073efb693951bd51420aa7456ef98c, ASSIGN in 373 msec 2023-03-10 18:53:40,250 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:53:40,251 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474420250"}]},"ts":"1678474420250"} 2023-03-10 18:53:40,253 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-10 18:53:40,256 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:53:40,259 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 843 msec 2023-03-10 18:53:40,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-10 18:53:40,541 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-10 18:53:40,562 DEBUG [hconnection-0x206d21f7-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:53:40,565 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52458, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:53:40,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:40,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:40,717 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:40,718 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:40,718 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-10 18:53:40,718 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-10 18:53:40,719 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:40,719 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:40,721 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:40,721 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:40,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:40,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:40,722 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:40,722 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:40,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:40,723 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:40,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:40,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:40,724 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:40,725 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:40,726 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-10 18:53:40,726 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:40,726 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-10 18:53:40,727 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:40,727 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:40,727 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:40,728 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. started... 2023-03-10 18:53:40,729 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a8073efb693951bd51420aa7456ef98c 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-10 18:53:41,296 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113003b324678c0e4c2dbf1b8bd3a1d12ddc to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113003b324678c0e4c2dbf1b8bd3a1d12ddc 2023-03-10 18:53:41,711 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/c841c14bf93a4de18d92b8d8ae575781 2023-03-10 18:53:41,721 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/c841c14bf93a4de18d92b8d8ae575781 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c841c14bf93a4de18d92b8d8ae575781 2023-03-10 18:53:41,730 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c841c14bf93a4de18d92b8d8ae575781, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-10 18:53:41,733 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for a8073efb693951bd51420aa7456ef98c in 1004ms, sequenceid=5, compaction requested=false 2023-03-10 18:53:41,735 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:41,738 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:41,738 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:41,738 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:41,739 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:41,739 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,739 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-10 18:53:41,739 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-10 18:53:41,741 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,741 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:41,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:41,742 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,742 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:41,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:41,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:41,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:41,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-10 18:53:41,744 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-10 18:53:41,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60ae93[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:41,744 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,746 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,746 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-10 18:53:41,746 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-10 18:53:41,746 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:41,746 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,746 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:41,749 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,749 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-10 18:53:41,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:41,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:41,749 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:41,749 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-10 18:53:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:41,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:41,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-10 18:53:41,752 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-10 18:53:41,753 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:41,753 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:41,753 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,753 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:41,756 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,756 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:41,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,756 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:41,757 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,758 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:41,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,766 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,766 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:41,766 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:41,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:41,766 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:41,766 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:41,766 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:41,766 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:41,766 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,767 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:41,767 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:41,767 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,767 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,768 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:41,768 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:41,869 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:41,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:41,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:41,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:41,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,886 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:41,886 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:41,887 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-10 18:53:41,887 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-10 18:53:41,887 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,887 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,889 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:41,889 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:41,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:41,889 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:41,889 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:41,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,890 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:41,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,890 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,891 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-10 18:53:41,891 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:41,891 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-10 18:53:41,891 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:41,891 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-10 18:53:41,891 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:41,891 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:41,891 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:41,891 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. started... 2023-03-10 18:53:41,892 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a8073efb693951bd51420aa7456ef98c 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-10 18:53:42,323 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ba0747866b474c28a7984064c1ac63d2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ba0747866b474c28a7984064c1ac63d2 2023-03-10 18:53:42,670 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-10 18:53:42,671 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-10 18:53:42,735 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/c6022c72f5a04c1184b715040cc6b82c 2023-03-10 18:53:42,749 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/c6022c72f5a04c1184b715040cc6b82c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c6022c72f5a04c1184b715040cc6b82c 2023-03-10 18:53:42,758 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c6022c72f5a04c1184b715040cc6b82c, entries=1, sequenceid=9, filesize=5.0 K 2023-03-10 18:53:42,760 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for a8073efb693951bd51420aa7456ef98c in 869ms, sequenceid=9, compaction requested=false 2023-03-10 18:53:42,760 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:42,760 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:42,760 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-10 18:53:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-10 18:53:42,763 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,763 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:42,764 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,764 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:42,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:42,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:42,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:42,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-10 18:53:42,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48a5fdfe[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:42,766 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-10 18:53:42,766 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,768 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,768 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-10 18:53:42,768 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-10 18:53:42,768 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:42,768 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,768 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:42,771 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,771 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-10 18:53:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:42,771 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:42,771 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-10 18:53:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:42,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:42,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-10 18:53:42,776 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-10 18:53:42,776 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:42,776 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:42,776 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,776 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:42,783 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,783 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:42,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,783 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:42,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:42,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:42,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:42,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:42,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,784 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:42,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,786 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:42,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,791 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,791 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,791 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:42,791 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:42,791 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:42,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:42,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:42,791 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:42,791 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,792 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:42,792 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,792 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:42,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:42,792 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,792 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:42,793 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:42,893 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:42,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:42,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:42,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:42,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,913 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:42,913 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:42,913 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-10 18:53:42,913 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-10 18:53:42,914 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,914 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,915 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,915 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:42,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:42,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:42,916 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:42,916 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:42,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,916 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:42,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,917 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,917 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-10 18:53:42,917 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:42,918 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-10 18:53:42,918 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:42,918 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-10 18:53:42,918 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:42,918 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:42,918 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:42,918 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. started... 2023-03-10 18:53:42,918 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a8073efb693951bd51420aa7456ef98c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:53:43,349 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201ec40ed1eae904cefa3e5cf6e95694ef2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201ec40ed1eae904cefa3e5cf6e95694ef2 2023-03-10 18:53:43,408 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:53:43,761 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/d4ee9de819054ab5ac58d60bbb412b5a 2023-03-10 18:53:43,770 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/d4ee9de819054ab5ac58d60bbb412b5a as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/d4ee9de819054ab5ac58d60bbb412b5a 2023-03-10 18:53:43,778 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/d4ee9de819054ab5ac58d60bbb412b5a, entries=1, sequenceid=13, filesize=5.0 K 2023-03-10 18:53:43,779 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a8073efb693951bd51420aa7456ef98c in 861ms, sequenceid=13, compaction requested=true 2023-03-10 18:53:43,779 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:43,779 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:43,779 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-10 18:53:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-10 18:53:43,783 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:43,784 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,784 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:43,786 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-10 18:53:43,786 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@847a922[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:43,786 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-10 18:53:43,786 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,787 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,787 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-10 18:53:43,787 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-10 18:53:43,787 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:43,788 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,788 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:43,789 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-10 18:53:43,789 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,789 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:43,789 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-10 18:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:43,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:43,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-10 18:53:43,793 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-10 18:53:43,793 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:43,793 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:43,793 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,793 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:43,795 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,795 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,795 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:43,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:43,796 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:43,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,798 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:43,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,803 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,803 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,803 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:43,803 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:43,803 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:43,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:43,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:43,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:43,803 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:43,803 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,804 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,804 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,804 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,804 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-10 18:53:43,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:43,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:43,804 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:43,805 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:43,902 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-10 18:53:43,902 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-10 18:53:43,905 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:43,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:43,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:43,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:43,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,917 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:43,917 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:43,917 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-10 18:53:43,918 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-10 18:53:43,918 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,918 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,920 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,920 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:43,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:43,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:43,920 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:43,920 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-10 18:53:43,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-10 18:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-10 18:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:43,922 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:43,923 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. started... 2023-03-10 18:53:43,923 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a8073efb693951bd51420aa7456ef98c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:53:44,347 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056ee2c9e8f7744662ba4371b408b992aa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512056ee2c9e8f7744662ba4371b408b992aa 2023-03-10 18:53:44,759 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/84710d61d9894867ac29afa7a24e8189 2023-03-10 18:53:44,769 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/84710d61d9894867ac29afa7a24e8189 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/84710d61d9894867ac29afa7a24e8189 2023-03-10 18:53:44,778 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/84710d61d9894867ac29afa7a24e8189, entries=1, sequenceid=17, filesize=5.0 K 2023-03-10 18:53:44,779 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a8073efb693951bd51420aa7456ef98c in 856ms, sequenceid=17, compaction requested=true 2023-03-10 18:53:44,779 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:44,779 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:44,779 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:44,780 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:44,780 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,780 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-10 18:53:44,780 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-10 18:53:44,782 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,782 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:44,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:44,783 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,783 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:44,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:44,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:44,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:44,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-10 18:53:44,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1433770d[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:44,785 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-10 18:53:44,785 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,786 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,787 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-10 18:53:44,787 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-10 18:53:44,787 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,787 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:44,787 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:44,790 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,790 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-10 18:53:44,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:44,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:44,790 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:44,790 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-10 18:53:44,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:44,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:44,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:44,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:44,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-10 18:53:44,794 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-10 18:53:44,794 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:44,794 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:44,794 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,794 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:44,796 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,796 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:44,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:44,796 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:44,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:44,797 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:44,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:44,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:44,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,799 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:44,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,804 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,804 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,804 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:44,804 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:44,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:44,804 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:44,805 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:44,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-10 18:53:44,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:44,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:44,805 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,805 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:44,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:44,805 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,806 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,806 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:44,806 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:44,906 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:44,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:44,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:44,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:44,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,926 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:44,927 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:44,927 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-10 18:53:44,927 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-10 18:53:44,928 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,928 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,929 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,929 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:44,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:44,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:44,930 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:44,930 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:44,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,930 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:44,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,931 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,931 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-10 18:53:44,931 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:44,931 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-10 18:53:44,932 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:44,932 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-10 18:53:44,933 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:44,933 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:44,933 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:44,933 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. started... 2023-03-10 18:53:44,933 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a8073efb693951bd51420aa7456ef98c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:53:45,362 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122806a501fa8a034418a0b174cf6bb281f6 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122806a501fa8a034418a0b174cf6bb281f6 2023-03-10 18:53:45,777 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/50e3bf3aa91d436c98d540a603a9ab03 2023-03-10 18:53:45,788 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/50e3bf3aa91d436c98d540a603a9ab03 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/50e3bf3aa91d436c98d540a603a9ab03 2023-03-10 18:53:45,797 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/50e3bf3aa91d436c98d540a603a9ab03, entries=1, sequenceid=21, filesize=5.0 K 2023-03-10 18:53:45,799 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a8073efb693951bd51420aa7456ef98c in 866ms, sequenceid=21, compaction requested=true 2023-03-10 18:53:45,799 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:45,799 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:45,799 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:45,800 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:45,800 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,800 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-10 18:53:45,800 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-10 18:53:45,805 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,805 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:45,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:45,805 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,805 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:45,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:45,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:45,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:45,807 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-10 18:53:45,807 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1122572e[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:45,807 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-10 18:53:45,808 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,810 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,810 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-10 18:53:45,810 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-10 18:53:45,810 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:45,810 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,810 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:45,812 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-10 18:53:45,812 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,812 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:45,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:45,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:45,812 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-10 18:53:45,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:45,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:45,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:45,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-10 18:53:45,817 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-10 18:53:45,817 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:45,817 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:45,817 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,817 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:45,826 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,826 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,826 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:45,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:45,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:45,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:45,827 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:45,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:45,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,829 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:45,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,833 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,833 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,833 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:45,833 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,833 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,833 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,833 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:45,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:45,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-10 18:53:45,834 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:45,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:45,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:45,833 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:45,834 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:45,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:45,835 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:45,835 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:45,935 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:45,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:45,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:45,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:45,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,948 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:45,948 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:45,948 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-10 18:53:45,949 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-10 18:53:45,949 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,949 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,951 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,951 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:45,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:45,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:45,952 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:45,952 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:45,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:45,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,953 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,954 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-10 18:53:45,954 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:45,954 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-10 18:53:45,954 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:45,954 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-10 18:53:45,954 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:45,954 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:45,954 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:45,955 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. started... 2023-03-10 18:53:45,955 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a8073efb693951bd51420aa7456ef98c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:53:46,426 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231869984ab1f304e0bb7537a12f6c8aa4f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231869984ab1f304e0bb7537a12f6c8aa4f 2023-03-10 18:53:46,835 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/7e19d151f77d4ed49a555f71c2d8e375 2023-03-10 18:53:46,845 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/7e19d151f77d4ed49a555f71c2d8e375 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/7e19d151f77d4ed49a555f71c2d8e375 2023-03-10 18:53:46,852 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/7e19d151f77d4ed49a555f71c2d8e375, entries=1, sequenceid=25, filesize=5.0 K 2023-03-10 18:53:46,853 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a8073efb693951bd51420aa7456ef98c in 898ms, sequenceid=25, compaction requested=true 2023-03-10 18:53:46,853 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:46,853 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:46,853 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:46,854 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:46,854 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,854 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-10 18:53:46,854 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-10 18:53:46,856 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,856 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:46,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:46,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:46,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-10 18:53:46,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@768f12f9[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:46,859 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-10 18:53:46,859 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,861 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-10 18:53:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-10 18:53:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:46,862 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,862 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:46,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-10 18:53:46,863 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:46,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-10 18:53:46,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:46,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:46,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:46,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:46,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:46,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:46,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-10 18:53:46,868 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-10 18:53:46,868 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:46,868 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:46,868 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,869 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:46,872 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,872 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:46,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:46,872 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:46,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:46,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:46,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:46,873 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:46,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,875 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:46,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,878 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,878 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,878 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,878 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:46,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-10 18:53:46,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:46,878 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:46,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:46,878 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:46,879 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:46,879 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:46,879 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,880 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,880 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:46,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:46,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:46,880 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:46,980 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:46,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:46,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:46,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:46,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,992 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:46,992 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:46,992 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-10 18:53:46,993 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-10 18:53:46,993 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,993 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,994 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,995 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:46,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:46,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:46,995 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:46,995 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:46,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:46,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,996 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-10 18:53:46,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:46,997 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-10 18:53:46,997 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:46,997 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-10 18:53:46,997 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:46,998 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:46,998 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:46,998 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. started... 2023-03-10 18:53:46,998 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a8073efb693951bd51420aa7456ef98c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:53:47,424 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101987a4996ec5d4f6f99fd0ccc34a793b6 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101987a4996ec5d4f6f99fd0ccc34a793b6 2023-03-10 18:53:47,833 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/aa7e539422bc47fe807e88a8c1eecb37 2023-03-10 18:53:47,841 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/aa7e539422bc47fe807e88a8c1eecb37 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa7e539422bc47fe807e88a8c1eecb37 2023-03-10 18:53:47,849 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa7e539422bc47fe807e88a8c1eecb37, entries=1, sequenceid=29, filesize=5.0 K 2023-03-10 18:53:47,851 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a8073efb693951bd51420aa7456ef98c in 853ms, sequenceid=29, compaction requested=true 2023-03-10 18:53:47,851 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:47,851 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:47,851 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:47,852 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:47,852 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,852 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-10 18:53:47,852 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-10 18:53:47,854 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,854 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:47,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:47,854 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,854 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:47,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:47,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:47,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:47,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-10 18:53:47,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43b4bbb3[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:47,856 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-10 18:53:47,856 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,858 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,858 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-10 18:53:47,858 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-10 18:53:47,858 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:47,858 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,858 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:47,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-10 18:53:47,860 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:47,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:47,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-10 18:53:47,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:47,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:47,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:47,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:47,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:47,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-10 18:53:47,862 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-10 18:53:47,863 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:47,863 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:47,863 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,863 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:47,864 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,864 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:47,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:47,864 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:47,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:47,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:47,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:47,865 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:47,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,866 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,873 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,873 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,873 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,873 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:47,873 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:47,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:47,873 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:47,873 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:47,873 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:47,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:47,873 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,874 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,874 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:47,874 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:47,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:47,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:47,974 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:47,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:47,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:47,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:47,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,986 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:47,986 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:47,987 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-10 18:53:47,987 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-10 18:53:47,988 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,988 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,989 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,989 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:47,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:47,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:47,989 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:47,989 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:47,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:47,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-10 18:53:47,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:47,991 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-10 18:53:47,991 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:47,992 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-10 18:53:47,992 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:47,992 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:47,992 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:47,992 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. started... 2023-03-10 18:53:47,992 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a8073efb693951bd51420aa7456ef98c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:53:48,018 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010342ff3e24a23348acbc49dd10db0dd5d4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010342ff3e24a23348acbc49dd10db0dd5d4 2023-03-10 18:53:48,027 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/34b9ef1bb3a74b418fd20250a9c2c442 2023-03-10 18:53:48,035 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/34b9ef1bb3a74b418fd20250a9c2c442 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/34b9ef1bb3a74b418fd20250a9c2c442 2023-03-10 18:53:48,043 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/34b9ef1bb3a74b418fd20250a9c2c442, entries=1, sequenceid=33, filesize=5.0 K 2023-03-10 18:53:48,044 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a8073efb693951bd51420aa7456ef98c in 52ms, sequenceid=33, compaction requested=true 2023-03-10 18:53:48,044 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:48,044 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:48,044 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:48,045 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:48,045 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,045 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-10 18:53:48,045 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-10 18:53:48,048 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,048 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:48,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:48,048 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,048 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:48,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:48,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:48,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:48,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-10 18:53:48,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4cb037d7[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:48,050 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-10 18:53:48,050 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,051 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,051 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-10 18:53:48,051 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-10 18:53:48,051 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:48,051 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,051 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:48,053 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,053 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-10 18:53:48,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:48,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:48,053 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:48,053 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-10 18:53:48,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:48,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:48,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:48,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:48,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-10 18:53:48,056 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-10 18:53:48,056 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:48,056 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:48,056 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,056 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:48,058 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,058 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:48,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:48,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,058 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:48,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:48,059 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:48,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:48,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:48,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,060 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:48,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,063 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,063 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,063 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:48,063 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:48,063 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:48,063 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:48,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:48,063 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:48,063 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,063 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:48,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,064 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:48,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:48,064 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-10 18:53:48,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:48,064 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:48,165 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:48,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:48,174 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:48,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:48,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,176 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:48,176 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:48,177 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-10 18:53:48,177 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-10 18:53:48,177 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,177 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,179 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,179 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:48,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:48,179 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,179 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,180 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:48,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,180 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,181 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-10 18:53:48,181 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,182 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-10 18:53:48,182 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:48,182 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-10 18:53:48,183 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:48,183 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:48,183 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:48,183 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. started... 2023-03-10 18:53:48,183 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a8073efb693951bd51420aa7456ef98c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:53:48,203 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509075bef9539d476463fb3c57b1de8d824f4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509075bef9539d476463fb3c57b1de8d824f4 2023-03-10 18:53:48,613 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/30982d7d5e6747c59017d60e171b4aee 2023-03-10 18:53:48,623 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/30982d7d5e6747c59017d60e171b4aee as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/30982d7d5e6747c59017d60e171b4aee 2023-03-10 18:53:48,631 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/30982d7d5e6747c59017d60e171b4aee, entries=1, sequenceid=37, filesize=5.0 K 2023-03-10 18:53:48,632 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a8073efb693951bd51420aa7456ef98c in 449ms, sequenceid=37, compaction requested=true 2023-03-10 18:53:48,633 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:48,633 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-10 18:53:48,633 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-10 18:53:48,635 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,635 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:48,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:48,636 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,636 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:48,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:48,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:48,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:48,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-10 18:53:48,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f288248[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:48,637 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-10 18:53:48,638 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,639 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,639 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-10 18:53:48,639 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-10 18:53:48,639 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:48,639 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,639 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:48,642 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-10 18:53:48,642 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,642 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:48,642 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-10 18:53:48,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:48,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:48,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:48,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:48,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-10 18:53:48,645 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-10 18:53:48,645 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:48,645 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:48,645 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,646 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:48,648 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,648 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:48,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,649 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:48,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:48,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:48,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:48,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:48,652 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,654 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:48,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:48,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:48,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:48,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:48,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-10 18:53:48,661 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:48,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,662 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,662 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,663 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,663 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:48,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:48,663 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:48,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:48,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:48,763 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:48,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:48,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:48,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:48,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,776 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:48,776 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:48,777 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-10 18:53:48,777 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-10 18:53:48,784 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,784 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,787 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:48,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:48,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:48,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,788 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,789 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:48,789 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:48,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:48,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,789 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,792 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-10 18:53:48,792 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:48,792 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-10 18:53:48,792 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:48,794 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-10 18:53:48,794 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:48,794 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:48,794 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:48,794 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. started... 2023-03-10 18:53:48,794 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a8073efb693951bd51420aa7456ef98c 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:53:49,242 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511205a432f96349546d7af8f105adaed8916 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511205a432f96349546d7af8f105adaed8916 2023-03-10 18:53:49,676 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/aa57adef07544a85b58eb33134222cf0 2023-03-10 18:53:49,683 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/aa57adef07544a85b58eb33134222cf0 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa57adef07544a85b58eb33134222cf0 2023-03-10 18:53:49,690 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa57adef07544a85b58eb33134222cf0, entries=1, sequenceid=41, filesize=5.0 K 2023-03-10 18:53:49,691 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a8073efb693951bd51420aa7456ef98c in 897ms, sequenceid=41, compaction requested=true 2023-03-10 18:53:49,691 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:49,691 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:49,691 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:49,691 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:49,691 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,692 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-10 18:53:49,692 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-10 18:53:49,693 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,693 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:49,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:49,694 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,694 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:49,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:49,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:49,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:49,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-10 18:53:49,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41be498e[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:49,695 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-10 18:53:49,695 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,697 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,697 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-10 18:53:49,698 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-10 18:53:49,698 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,698 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:49,698 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:49,701 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,701 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-10 18:53:49,702 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:49,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:49,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:49,702 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-10 18:53:49,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:49,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:49,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:49,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:49,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-10 18:53:49,706 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-10 18:53:49,706 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:49,706 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:49,706 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,706 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:49,707 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,708 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:49,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:49,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,708 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:49,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:49,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:49,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:49,708 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:49,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,710 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:49,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,718 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,718 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:49,718 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:49,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-10 18:53:49,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-10 18:53:49,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:49,718 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,718 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:49,718 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:49,718 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:49,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:49,718 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,719 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,719 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,719 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:49,719 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:49,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:49,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:49,819 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:49,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:49,825 DEBUG [Listener at localhost/36413] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-10 18:53:49,828 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53904, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-10 18:53:49,829 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:49,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-10 18:53:49,842 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-10 18:53:49,844 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:53:49,844 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475029844 2023-03-10 18:53:49,846 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:53:49,850 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-10 18:53:49,850 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-10 18:53:49,850 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x67868145 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:53:49,854 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@61a30a88, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:53:49,856 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:50,324 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031b16ba4cdecb4fd2a69cbc70f211998e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601031b16ba4cdecb4fd2a69cbc70f211998e 2023-03-10 18:53:50,325 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-10 18:53:50,343 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:53:50,345 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56112, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:53:50,355 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:53:50,357 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53912, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:53:50,389 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3a3bd13600c5451c981490ec52f69fdd first=Optional[r3] last=Optional[r6] 2023-03-10 18:53:50,401 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3a3bd13600c5451c981490ec52f69fdd}] 2023-03-10 18:53:50,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3a3bd13600c5451c981490ec52f69fdd for inclusion in a8073efb693951bd51420aa7456ef98c/family1 2023-03-10 18:53:50,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-10 18:53:50,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:53:50,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:50,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3a3bd13600c5451c981490ec52f69fdd to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ovcfsuij4imagrbosc1aq8t6qvtre1f87vfqa5h94vfb052h3ha4ebne8gtag52a/family1/3a3bd13600c5451c981490ec52f69fdd 2023-03-10 18:53:50,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ovcfsuij4imagrbosc1aq8t6qvtre1f87vfqa5h94vfb052h3ha4ebne8gtag52a/family1/3a3bd13600c5451c981490ec52f69fdd as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/a633afd14cc940f28a30edb5f5fd1f38_SeqId_44_ 2023-03-10 18:53:50,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ovcfsuij4imagrbosc1aq8t6qvtre1f87vfqa5h94vfb052h3ha4ebne8gtag52a/family1/3a3bd13600c5451c981490ec52f69fdd into a8073efb693951bd51420aa7456ef98c/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/a633afd14cc940f28a30edb5f5fd1f38_SeqId_44_ - updating store file list. 2023-03-10 18:53:50,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/a633afd14cc940f28a30edb5f5fd1f38_SeqId_44_ into a8073efb693951bd51420aa7456ef98c/family1 2023-03-10 18:53:50,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ovcfsuij4imagrbosc1aq8t6qvtre1f87vfqa5h94vfb052h3ha4ebne8gtag52a/family1/3a3bd13600c5451c981490ec52f69fdd into a8073efb693951bd51420aa7456ef98c/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/a633afd14cc940f28a30edb5f5fd1f38_SeqId_44_) 2023-03-10 18:53:50,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ovcfsuij4imagrbosc1aq8t6qvtre1f87vfqa5h94vfb052h3ha4ebne8gtag52a/family1/3a3bd13600c5451c981490ec52f69fdd 2023-03-10 18:53:50,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-10 18:53:50,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-10 18:53:50,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): a8073efb693951bd51420aa7456ef98c/family1 is initiating minor compaction 2023-03-10 18:53:50,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@37c0e061; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:53:50,446 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a8073efb693951bd51420aa7456ef98c/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:50,447 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c6022c72f5a04c1184b715040cc6b82c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/d4ee9de819054ab5ac58d60bbb412b5a, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/84710d61d9894867ac29afa7a24e8189, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/50e3bf3aa91d436c98d540a603a9ab03, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/7e19d151f77d4ed49a555f71c2d8e375, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa7e539422bc47fe807e88a8c1eecb37, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/34b9ef1bb3a74b418fd20250a9c2c442, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/30982d7d5e6747c59017d60e171b4aee, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa57adef07544a85b58eb33134222cf0, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/a633afd14cc940f28a30edb5f5fd1f38_SeqId_44_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp, totalSize=50.2 K 2023-03-10 18:53:50,448 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting c6022c72f5a04c1184b715040cc6b82c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-10 18:53:50,448 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting d4ee9de819054ab5ac58d60bbb412b5a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-10 18:53:50,448 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 84710d61d9894867ac29afa7a24e8189, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-10 18:53:50,448 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 50e3bf3aa91d436c98d540a603a9ab03, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-10 18:53:50,448 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 7e19d151f77d4ed49a555f71c2d8e375, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-10 18:53:50,448 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting aa7e539422bc47fe807e88a8c1eecb37, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-10 18:53:50,448 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 34b9ef1bb3a74b418fd20250a9c2c442, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-10 18:53:50,448 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 30982d7d5e6747c59017d60e171b4aee, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-10 18:53:50,448 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting aa57adef07544a85b58eb33134222cf0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-10 18:53:50,448 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting a633afd14cc940f28a30edb5f5fd1f38_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-10 18:53:50,460 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-10 18:53:50,462 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:53:50,467 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122806a501fa8a034418a0b174cf6bb281f6 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122806a501fa8a034418a0b174cf6bb281f6 2023-03-10 18:53:50,469 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231869984ab1f304e0bb7537a12f6c8aa4f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231869984ab1f304e0bb7537a12f6c8aa4f 2023-03-10 18:53:50,471 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101987a4996ec5d4f6f99fd0ccc34a793b6 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101987a4996ec5d4f6f99fd0ccc34a793b6 2023-03-10 18:53:50,473 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010342ff3e24a23348acbc49dd10db0dd5d4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010342ff3e24a23348acbc49dd10db0dd5d4 2023-03-10 18:53:50,473 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-10 18:53:50,473 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-10 18:53:50,474 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:50,474 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:50,474 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:50,474 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:50,474 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:50,474 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:50,474 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:50,474 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:50,504 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): a8073efb693951bd51420aa7456ef98c#family1#compaction#10 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:53:51,333 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/e3736f42208a43b197dfc0adbc7de7ba as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/e3736f42208a43b197dfc0adbc7de7ba 2023-03-10 18:53:51,349 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in a8073efb693951bd51420aa7456ef98c/family1 of a8073efb693951bd51420aa7456ef98c into e3736f42208a43b197dfc0adbc7de7ba(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:53:51,349 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:51,350 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1678474430445; duration=0sec 2023-03-10 18:53:51,351 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:53:51,366 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205432770eab9d04a9a93a5cb6a964a1c1e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205432770eab9d04a9a93a5cb6a964a1c1e 2023-03-10 18:53:51,368 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-10 18:53:51,398 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/59301ebb10ff45c48d7955a7e79f669f first=Optional[r00] last=Optional[r2] 2023-03-10 18:53:51,399 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/59301ebb10ff45c48d7955a7e79f669f}] 2023-03-10 18:53:51,402 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/59301ebb10ff45c48d7955a7e79f669f for inclusion in a8073efb693951bd51420aa7456ef98c/family1 2023-03-10 18:53:51,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-10 18:53:51,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:53:51,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:51,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/59301ebb10ff45c48d7955a7e79f669f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__l3ddss0fsc75kug416ose4lnfvi0q3les2h06r0s7djegjmhm6gakp38o81en7uu/family1/59301ebb10ff45c48d7955a7e79f669f 2023-03-10 18:53:51,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__l3ddss0fsc75kug416ose4lnfvi0q3les2h06r0s7djegjmhm6gakp38o81en7uu/family1/59301ebb10ff45c48d7955a7e79f669f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/ef2361ca9cb1418681244afa0984ef0c_SeqId_47_ 2023-03-10 18:53:51,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__l3ddss0fsc75kug416ose4lnfvi0q3les2h06r0s7djegjmhm6gakp38o81en7uu/family1/59301ebb10ff45c48d7955a7e79f669f into a8073efb693951bd51420aa7456ef98c/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/ef2361ca9cb1418681244afa0984ef0c_SeqId_47_ - updating store file list. 2023-03-10 18:53:51,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/ef2361ca9cb1418681244afa0984ef0c_SeqId_47_ into a8073efb693951bd51420aa7456ef98c/family1 2023-03-10 18:53:51,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__l3ddss0fsc75kug416ose4lnfvi0q3les2h06r0s7djegjmhm6gakp38o81en7uu/family1/59301ebb10ff45c48d7955a7e79f669f into a8073efb693951bd51420aa7456ef98c/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/ef2361ca9cb1418681244afa0984ef0c_SeqId_47_) 2023-03-10 18:53:51,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__l3ddss0fsc75kug416ose4lnfvi0q3les2h06r0s7djegjmhm6gakp38o81en7uu/family1/59301ebb10ff45c48d7955a7e79f669f 2023-03-10 18:53:51,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-10 18:53:51,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-10 18:53:51,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): a8073efb693951bd51420aa7456ef98c/family1 is initiating minor compaction (all files) 2023-03-10 18:53:51,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@28af40aa; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:53:51,425 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a8073efb693951bd51420aa7456ef98c/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:51,426 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c841c14bf93a4de18d92b8d8ae575781, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/e3736f42208a43b197dfc0adbc7de7ba, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/ef2361ca9cb1418681244afa0984ef0c_SeqId_47_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp, totalSize=358.6 K 2023-03-10 18:53:51,426 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c6022c72f5a04c1184b715040cc6b82c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/d4ee9de819054ab5ac58d60bbb412b5a, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/84710d61d9894867ac29afa7a24e8189, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/50e3bf3aa91d436c98d540a603a9ab03, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/7e19d151f77d4ed49a555f71c2d8e375, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa7e539422bc47fe807e88a8c1eecb37, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/34b9ef1bb3a74b418fd20250a9c2c442, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/30982d7d5e6747c59017d60e171b4aee, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa57adef07544a85b58eb33134222cf0, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/a633afd14cc940f28a30edb5f5fd1f38_SeqId_44_] to archive 2023-03-10 18:53:51,427 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting c841c14bf93a4de18d92b8d8ae575781, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839360 2023-03-10 18:53:51,428 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:53:51,429 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting e3736f42208a43b197dfc0adbc7de7ba, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239360 2023-03-10 18:53:51,430 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting ef2361ca9cb1418681244afa0984ef0c_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878839360 2023-03-10 18:53:51,430 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-10 18:53:51,431 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:53:51,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c6022c72f5a04c1184b715040cc6b82c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c6022c72f5a04c1184b715040cc6b82c 2023-03-10 18:53:51,433 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/d4ee9de819054ab5ac58d60bbb412b5a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/d4ee9de819054ab5ac58d60bbb412b5a 2023-03-10 18:53:51,434 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113003b324678c0e4c2dbf1b8bd3a1d12ddc to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113003b324678c0e4c2dbf1b8bd3a1d12ddc 2023-03-10 18:53:51,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/84710d61d9894867ac29afa7a24e8189 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/84710d61d9894867ac29afa7a24e8189 2023-03-10 18:53:51,439 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201ec40ed1eae904cefa3e5cf6e95694ef2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201ec40ed1eae904cefa3e5cf6e95694ef2 2023-03-10 18:53:51,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/50e3bf3aa91d436c98d540a603a9ab03 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/50e3bf3aa91d436c98d540a603a9ab03 2023-03-10 18:53:51,442 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512056ee2c9e8f7744662ba4371b408b992aa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512056ee2c9e8f7744662ba4371b408b992aa 2023-03-10 18:53:51,442 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-10 18:53:51,442 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-10 18:53:51,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/7e19d151f77d4ed49a555f71c2d8e375 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/7e19d151f77d4ed49a555f71c2d8e375 2023-03-10 18:53:51,442 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:51,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-10 18:53:51,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:53:51,444 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-10 18:53:51,445 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa7e539422bc47fe807e88a8c1eecb37 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa7e539422bc47fe807e88a8c1eecb37 2023-03-10 18:53:51,448 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/34b9ef1bb3a74b418fd20250a9c2c442 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/34b9ef1bb3a74b418fd20250a9c2c442 2023-03-10 18:53:51,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.6100 sec 2023-03-10 18:53:51,449 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/30982d7d5e6747c59017d60e171b4aee to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/30982d7d5e6747c59017d60e171b4aee 2023-03-10 18:53:51,451 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa57adef07544a85b58eb33134222cf0 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/aa57adef07544a85b58eb33134222cf0 2023-03-10 18:53:51,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/a633afd14cc940f28a30edb5f5fd1f38_SeqId_44_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/a633afd14cc940f28a30edb5f5fd1f38_SeqId_44_ 2023-03-10 18:53:51,474 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): a8073efb693951bd51420aa7456ef98c#family1#compaction#11 average throughput is 6.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:53:51,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-10 18:53:51,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:51,643 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-10 18:53:51,645 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-10 18:53:51,646 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-10 18:53:51,652 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-10 18:53:51,670 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a8073efb693951bd51420aa7456ef98c, REOPEN/MOVE}] 2023-03-10 18:53:51,671 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a8073efb693951bd51420aa7456ef98c, REOPEN/MOVE 2023-03-10 18:53:51,672 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=a8073efb693951bd51420aa7456ef98c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:51,672 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678474431672"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474431672"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474431672"}]},"ts":"1678474431672"} 2023-03-10 18:53:51,674 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure a8073efb693951bd51420aa7456ef98c, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:53:51,833 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:51,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a8073efb693951bd51420aa7456ef98c, disabling compactions & flushes 2023-03-10 18:53:51,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:52,727 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/.tmp/family1/4be358adcdb64ad2a126f23ac925b7cc as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/4be358adcdb64ad2a126f23ac925b7cc 2023-03-10 18:53:52,734 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a8073efb693951bd51420aa7456ef98c/family1 of a8073efb693951bd51420aa7456ef98c into 4be358adcdb64ad2a126f23ac925b7cc(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:53:52,734 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:52,734 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:52,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:52,734 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1678474431424; duration=1sec 2023-03-10 18:53:52,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. after waiting 0 ms 2023-03-10 18:53:52,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:52,734 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:53:52,737 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c841c14bf93a4de18d92b8d8ae575781, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/e3736f42208a43b197dfc0adbc7de7ba, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/ef2361ca9cb1418681244afa0984ef0c_SeqId_47_] to archive 2023-03-10 18:53:52,738 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:53:52,740 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c841c14bf93a4de18d92b8d8ae575781 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/c841c14bf93a4de18d92b8d8ae575781 2023-03-10 18:53:52,742 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/e3736f42208a43b197dfc0adbc7de7ba to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/e3736f42208a43b197dfc0adbc7de7ba 2023-03-10 18:53:52,744 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/ef2361ca9cb1418681244afa0984ef0c_SeqId_47_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/ef2361ca9cb1418681244afa0984ef0c_SeqId_47_ 2023-03-10 18:53:52,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-10 18:53:52,752 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:52,752 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:52,752 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: a8073efb693951bd51420aa7456ef98c to self. 2023-03-10 18:53:52,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:52,755 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=a8073efb693951bd51420aa7456ef98c, regionState=CLOSED 2023-03-10 18:53:52,755 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678474432755"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474432755"}]},"ts":"1678474432755"} 2023-03-10 18:53:52,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-10 18:53:52,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure a8073efb693951bd51420aa7456ef98c, server=jenkins-hbase4.apache.org,43181,1678474414959 in 1.0820 sec 2023-03-10 18:53:52,759 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a8073efb693951bd51420aa7456ef98c, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=true 2023-03-10 18:53:52,910 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=a8073efb693951bd51420aa7456ef98c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:52,910 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678474432910"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474432910"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474432910"}]},"ts":"1678474432910"} 2023-03-10 18:53:52,913 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure a8073efb693951bd51420aa7456ef98c, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:53:53,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:53,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a8073efb693951bd51420aa7456ef98c, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:53:53,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:53,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:53:53,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:53,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:53,071 INFO [StoreOpener-a8073efb693951bd51420aa7456ef98c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:53,072 DEBUG [StoreOpener-a8073efb693951bd51420aa7456ef98c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1 2023-03-10 18:53:53,072 DEBUG [StoreOpener-a8073efb693951bd51420aa7456ef98c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1 2023-03-10 18:53:53,072 INFO [StoreOpener-a8073efb693951bd51420aa7456ef98c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a8073efb693951bd51420aa7456ef98c columnFamilyName family1 2023-03-10 18:53:53,079 DEBUG [StoreOpener-a8073efb693951bd51420aa7456ef98c-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/4be358adcdb64ad2a126f23ac925b7cc 2023-03-10 18:53:53,080 INFO [StoreOpener-a8073efb693951bd51420aa7456ef98c-1] regionserver.HStore(310): Store=a8073efb693951bd51420aa7456ef98c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:53:53,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:53,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:53,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a8073efb693951bd51420aa7456ef98c 2023-03-10 18:53:53,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a8073efb693951bd51420aa7456ef98c; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74333005, jitterRate=0.1076480895280838}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:53:53,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:53,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c., pid=17, masterSystemTime=1678474433065 2023-03-10 18:53:53,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:53,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:53:53,090 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=a8073efb693951bd51420aa7456ef98c, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:53,090 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678474433090"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474433090"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474433090"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474433090"}]},"ts":"1678474433090"} 2023-03-10 18:53:53,094 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-10 18:53:53,094 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure a8073efb693951bd51420aa7456ef98c, server=jenkins-hbase4.apache.org,43181,1678474414959 in 180 msec 2023-03-10 18:53:53,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-10 18:53:53,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a8073efb693951bd51420aa7456ef98c, REOPEN/MOVE in 1.4250 sec 2023-03-10 18:53:53,099 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-10 18:53:53,099 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.4470 sec 2023-03-10 18:53:53,102 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.4870 sec 2023-03-10 18:53:53,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-10 18:53:53,112 INFO [Listener at localhost/36413] compactions.TestMobCompactor(278): alter status finished 2023-03-10 18:53:53,113 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-10 18:53:53,114 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-10 18:53:53,115 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-10 18:53:53,117 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:53:53,117 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475033117 2023-03-10 18:53:53,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:53:53,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-10 18:53:53,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-10 18:53:53,122 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x38a3fd56 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:53:53,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@53ca8242, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:53:53,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:53,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:53,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:53,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:53,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:53,970 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128c0349480e9444e60897a4f4a8aca59a8 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128c0349480e9444e60897a4f4a8aca59a8 2023-03-10 18:53:53,971 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-10 18:53:53,984 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:53:53,985 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42232, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:53:53,993 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:53:53,995 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53088, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:53:54,005 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8bf5402fc4af411dab04467a44590ec1 first=Optional[r05] last=Optional[r8] 2023-03-10 18:53:54,006 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=51 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8bf5402fc4af411dab04467a44590ec1}] 2023-03-10 18:53:54,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8bf5402fc4af411dab04467a44590ec1 for inclusion in a8073efb693951bd51420aa7456ef98c/family1 2023-03-10 18:53:54,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-10 18:53:54,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:53:54,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:53:54,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8bf5402fc4af411dab04467a44590ec1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__f7he68qp0hgab5acs43tt0trhu8rtv614vv6d0s3sba8vl6qfg1v2n3ts6gqfnpr/family1/8bf5402fc4af411dab04467a44590ec1 2023-03-10 18:53:54,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__f7he68qp0hgab5acs43tt0trhu8rtv614vv6d0s3sba8vl6qfg1v2n3ts6gqfnpr/family1/8bf5402fc4af411dab04467a44590ec1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/8fa255b7113c4d018dcbf647dffd2132_SeqId_53_ 2023-03-10 18:53:54,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__f7he68qp0hgab5acs43tt0trhu8rtv614vv6d0s3sba8vl6qfg1v2n3ts6gqfnpr/family1/8bf5402fc4af411dab04467a44590ec1 into a8073efb693951bd51420aa7456ef98c/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/8fa255b7113c4d018dcbf647dffd2132_SeqId_53_ - updating store file list. 2023-03-10 18:53:54,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/8fa255b7113c4d018dcbf647dffd2132_SeqId_53_ into a8073efb693951bd51420aa7456ef98c/family1 2023-03-10 18:53:54,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__f7he68qp0hgab5acs43tt0trhu8rtv614vv6d0s3sba8vl6qfg1v2n3ts6gqfnpr/family1/8bf5402fc4af411dab04467a44590ec1 into a8073efb693951bd51420aa7456ef98c/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/family1/8fa255b7113c4d018dcbf647dffd2132_SeqId_53_) 2023-03-10 18:53:54,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__f7he68qp0hgab5acs43tt0trhu8rtv614vv6d0s3sba8vl6qfg1v2n3ts6gqfnpr/family1/8bf5402fc4af411dab04467a44590ec1 2023-03-10 18:53:54,028 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-10 18:53:54,029 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:53:54,031 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511205a432f96349546d7af8f105adaed8916 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511205a432f96349546d7af8f105adaed8916 2023-03-10 18:53:54,032 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ba0747866b474c28a7984064c1ac63d2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ba0747866b474c28a7984064c1ac63d2 2023-03-10 18:53:54,032 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-10 18:53:54,033 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-10 18:53:54,033 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:54,033 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:54,033 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-10 18:53:54,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-10 18:53:54,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:53:54,034 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-10 18:53:54,036 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 921 msec 2023-03-10 18:53:54,125 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=248 (was 211) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56431@0x67868145-SendThread(127.0.0.1:56431) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56431@0x38a3fd56-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x60f15f3b-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-922730382_15 at /127.0.0.1:59948 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:59988 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:59998 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:59810 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/cluster_7b2047bb-341b-b20b-9460-0c7ebaa01809/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7809b0ca-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/cluster_7b2047bb-341b-b20b-9460-0c7ebaa01809/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:59876 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:43181-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56431@0x38a3fd56-SendThread(127.0.0.1:56431) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56431@0x67868145 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/229095779.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:59862 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:59780 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7809b0ca-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56431@0x38a3fd56 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/229095779.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:59972 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:59956 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x60f15f3b-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:59836 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:59758 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:60004 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7809b0ca-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56431@0x67868145-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:59898 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:60050 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=592 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=281 (was 313), ProcessCount=170 (was 170), AvailableMemoryMB=5617 (was 5974) 2023-03-10 18:53:54,140 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=592, MaxFileDescriptor=60000, SystemLoadAverage=281, ProcessCount=170, AvailableMemoryMB=5616 2023-03-10 18:53:54,142 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-10 18:53:54,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:54,145 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:53:54,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-10 18:53:54,145 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:53:54,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-10 18:53:54,147 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b 2023-03-10 18:53:54,148 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b empty. 2023-03-10 18:53:54,148 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b 2023-03-10 18:53:54,148 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-10 18:53:54,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-10 18:53:54,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-10 18:53:54,558 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-10 18:53:54,559 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 01b058e52c536aa30e6d0bad142a2e3b, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:53:54,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-10 18:53:54,966 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:53:54,967 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 01b058e52c536aa30e6d0bad142a2e3b, disabling compactions & flushes 2023-03-10 18:53:54,967 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:54,967 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:54,967 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. after waiting 0 ms 2023-03-10 18:53:54,967 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:54,967 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:54,967 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:53:54,972 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:53:54,974 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678474434974"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474434974"}]},"ts":"1678474434974"} 2023-03-10 18:53:54,977 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-10 18:53:54,978 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:53:54,978 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474434978"}]},"ts":"1678474434978"} 2023-03-10 18:53:54,981 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-10 18:53:54,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=01b058e52c536aa30e6d0bad142a2e3b, ASSIGN}] 2023-03-10 18:53:54,989 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=01b058e52c536aa30e6d0bad142a2e3b, ASSIGN 2023-03-10 18:53:54,991 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=01b058e52c536aa30e6d0bad142a2e3b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:53:55,142 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=01b058e52c536aa30e6d0bad142a2e3b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:55,142 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678474435142"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474435142"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474435142"}]},"ts":"1678474435142"} 2023-03-10 18:53:55,145 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 01b058e52c536aa30e6d0bad142a2e3b, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:53:55,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-10 18:53:55,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:55,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 01b058e52c536aa30e6d0bad142a2e3b, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:53:55,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 01b058e52c536aa30e6d0bad142a2e3b 2023-03-10 18:53:55,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:53:55,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 01b058e52c536aa30e6d0bad142a2e3b 2023-03-10 18:53:55,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 01b058e52c536aa30e6d0bad142a2e3b 2023-03-10 18:53:55,303 INFO [StoreOpener-01b058e52c536aa30e6d0bad142a2e3b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 01b058e52c536aa30e6d0bad142a2e3b 2023-03-10 18:53:55,305 DEBUG [StoreOpener-01b058e52c536aa30e6d0bad142a2e3b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1 2023-03-10 18:53:55,305 DEBUG [StoreOpener-01b058e52c536aa30e6d0bad142a2e3b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1 2023-03-10 18:53:55,305 INFO [StoreOpener-01b058e52c536aa30e6d0bad142a2e3b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 01b058e52c536aa30e6d0bad142a2e3b columnFamilyName family1 2023-03-10 18:53:55,307 INFO [StoreOpener-01b058e52c536aa30e6d0bad142a2e3b-1] regionserver.HStore(310): Store=01b058e52c536aa30e6d0bad142a2e3b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:53:55,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b 2023-03-10 18:53:55,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b 2023-03-10 18:53:55,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 01b058e52c536aa30e6d0bad142a2e3b 2023-03-10 18:53:55,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:53:55,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 01b058e52c536aa30e6d0bad142a2e3b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60520995, jitterRate=-0.09816689789295197}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:53:55,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:53:55,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b., pid=21, masterSystemTime=1678474435297 2023-03-10 18:53:55,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:55,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:55,317 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=01b058e52c536aa30e6d0bad142a2e3b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:55,317 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678474435317"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474435317"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474435317"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474435317"}]},"ts":"1678474435317"} 2023-03-10 18:53:55,320 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-10 18:53:55,320 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 01b058e52c536aa30e6d0bad142a2e3b, server=jenkins-hbase4.apache.org,43181,1678474414959 in 173 msec 2023-03-10 18:53:55,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-10 18:53:55,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=01b058e52c536aa30e6d0bad142a2e3b, ASSIGN in 334 msec 2023-03-10 18:53:55,323 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:53:55,323 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474435323"}]},"ts":"1678474435323"} 2023-03-10 18:53:55,324 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-10 18:53:55,327 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:53:55,328 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.1850 sec 2023-03-10 18:53:56,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-10 18:53:56,252 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-10 18:53:56,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:56,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:56,288 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:53:56,288 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:56,288 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:53:56,289 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:53:56,289 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:56,289 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:56,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:56,290 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:56,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:56,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:56,291 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:56,291 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:56,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:56,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:56,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:56,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:56,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:56,292 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:56,292 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:53:56,292 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:56,292 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:53:56,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:56,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:56,293 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:56,293 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. started... 2023-03-10 18:53:56,293 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 01b058e52c536aa30e6d0bad142a2e3b 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-10 18:53:56,717 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511307bb86a9570c848a2be35c063da1c73d1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511307bb86a9570c848a2be35c063da1c73d1 2023-03-10 18:53:57,121 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/6832806035aa42bf9fa14e3c8ae502b8 2023-03-10 18:53:57,128 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/6832806035aa42bf9fa14e3c8ae502b8 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/6832806035aa42bf9fa14e3c8ae502b8 2023-03-10 18:53:57,134 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/6832806035aa42bf9fa14e3c8ae502b8, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-10 18:53:57,135 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 01b058e52c536aa30e6d0bad142a2e3b in 842ms, sequenceid=5, compaction requested=false 2023-03-10 18:53:57,135 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:53:57,136 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:53:57,136 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:57,136 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:57,136 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:57,136 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,136 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:53:57,136 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:53:57,191 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,191 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:57,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:57,192 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,192 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:57,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:57,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:57,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:57,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:53:57,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27b11b29[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:57,194 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:53:57,194 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,222 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:53:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:53:57,223 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,223 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:57,223 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:57,268 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,268 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:53:57,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:57,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:57,268 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:57,269 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:53:57,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:57,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:53:57,272 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:53:57,272 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:57,272 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:57,272 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,272 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:57,322 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,322 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:57,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:57,322 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:57,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:57,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:57,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:57,324 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:57,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,325 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:57,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,376 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,376 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:57,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:53:57,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:57,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:57,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:57,376 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:57,376 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:57,376 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,377 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:57,378 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:57,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:57,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:57,478 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:57,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:57,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:57,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:57,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,486 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:53:57,486 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:57,487 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:53:57,487 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:53:57,488 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,488 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,545 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,545 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:57,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:57,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:57,546 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:57,546 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:57,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,546 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:57,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,547 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,547 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:53:57,547 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:57,547 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:53:57,547 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:57,548 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:53:57,548 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:57,548 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:57,548 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:57,548 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. started... 2023-03-10 18:53:57,548 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 01b058e52c536aa30e6d0bad142a2e3b 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-10 18:53:57,963 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a2deb52f0c474882987f8fafe18adbe3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128a2deb52f0c474882987f8fafe18adbe3 2023-03-10 18:53:58,287 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:53:58,370 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/5eaf0c695c48495e9e02603549a5937f 2023-03-10 18:53:58,377 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/5eaf0c695c48495e9e02603549a5937f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/5eaf0c695c48495e9e02603549a5937f 2023-03-10 18:53:58,384 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/5eaf0c695c48495e9e02603549a5937f, entries=1, sequenceid=9, filesize=5.0 K 2023-03-10 18:53:58,386 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 01b058e52c536aa30e6d0bad142a2e3b in 837ms, sequenceid=9, compaction requested=false 2023-03-10 18:53:58,386 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:53:58,386 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:58,386 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:58,386 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:58,386 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,386 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:53:58,386 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:53:58,389 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,389 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:58,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:58,389 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,389 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:58,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:58,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:58,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:58,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:53:58,391 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:53:58,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a728e87[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:58,391 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,393 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:53:58,393 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,393 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:58,394 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,394 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:53:58,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:58,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:58,395 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:58,395 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:53:58,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:58,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:58,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:58,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:58,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:53:58,398 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:53:58,398 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:58,399 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:58,399 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,399 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:58,400 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,400 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:58,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:58,401 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:58,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:58,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:58,401 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:58,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:58,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:58,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,408 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,416 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,416 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:58,417 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:53:58,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:58,417 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:58,417 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:58,417 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:58,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:58,417 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,418 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:58,418 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,418 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:58,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:58,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:58,418 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,418 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,518 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:58,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:58,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:58,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:58,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,527 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:53:58,527 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:58,528 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:53:58,528 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:53:58,529 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,529 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,530 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:58,530 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:58,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:58,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,531 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,531 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:58,531 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:58,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,537 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:53:58,537 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,537 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:53:58,537 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:58,538 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:53:58,539 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:58,539 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:58,539 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:58,539 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. started... 2023-03-10 18:53:58,539 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 01b058e52c536aa30e6d0bad142a2e3b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:53:58,565 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201c5096d21eb964cae9501ec07dea2fbc4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201c5096d21eb964cae9501ec07dea2fbc4 2023-03-10 18:53:58,971 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/912e37e5b3a64a5e94b3c893a4f0e538 2023-03-10 18:53:58,980 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/912e37e5b3a64a5e94b3c893a4f0e538 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/912e37e5b3a64a5e94b3c893a4f0e538 2023-03-10 18:53:58,986 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/912e37e5b3a64a5e94b3c893a4f0e538, entries=1, sequenceid=13, filesize=4.9 K 2023-03-10 18:53:58,988 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 01b058e52c536aa30e6d0bad142a2e3b in 449ms, sequenceid=13, compaction requested=true 2023-03-10 18:53:58,988 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:53:58,988 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:58,988 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:58,988 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:58,988 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,988 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:53:58,988 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:53:58,990 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,990 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:58,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:58,991 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,991 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:58,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:53:58,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61437b43[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:58,993 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:53:58,993 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,994 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,994 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:53:58,994 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:53:58,994 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:58,994 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,994 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:58,997 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:53:58,997 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:58,997 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,997 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:59,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:59,001 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:53:59,001 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:53:59,001 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:59,001 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:59,001 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,001 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:59,002 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,002 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,002 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:59,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:59,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:59,003 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:59,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,005 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,007 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,007 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,007 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,007 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,007 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,007 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,007 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,008 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:59,008 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:59,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:59,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:59,008 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:59,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:53:59,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:59,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:59,009 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:59,009 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:53:59,109 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:53:59,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:53:59,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:53:59,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:53:59,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,119 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:53:59,119 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:59,119 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:53:59,119 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:53:59,120 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,120 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,121 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:59,121 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:59,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:59,122 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,122 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:53:59,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:53:59,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,123 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,123 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:53:59,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,123 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:53:59,123 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:53:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:53:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:53:59,124 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:59,124 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. started... 2023-03-10 18:53:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:53:59,124 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 01b058e52c536aa30e6d0bad142a2e3b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:53:59,540 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f8a72ce024da48f184fb71d7d2f9c75a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f8a72ce024da48f184fb71d7d2f9c75a 2023-03-10 18:53:59,946 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/54ef74abf72b418eb8b66381698164aa 2023-03-10 18:53:59,952 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/54ef74abf72b418eb8b66381698164aa as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/54ef74abf72b418eb8b66381698164aa 2023-03-10 18:53:59,957 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/54ef74abf72b418eb8b66381698164aa, entries=1, sequenceid=17, filesize=4.9 K 2023-03-10 18:53:59,958 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 01b058e52c536aa30e6d0bad142a2e3b in 834ms, sequenceid=17, compaction requested=true 2023-03-10 18:53:59,959 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:53:59,959 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:53:59,959 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:53:59,959 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:53:59,959 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,959 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:53:59,959 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:53:59,962 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,962 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:59,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:59,962 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,962 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:53:59,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:59,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:59,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:59,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:53:59,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5282bcdc[Count = 0] remaining members to acquire global barrier 2023-03-10 18:53:59,964 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:53:59,964 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,965 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,966 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:53:59,966 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:53:59,966 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:53:59,966 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,966 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:53:59,967 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,967 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:53:59,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,967 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:59,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:59,967 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:53:59,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:59,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:53:59,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:53:59,970 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:53:59,970 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:53:59,970 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:53:59,970 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,970 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:53:59,972 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,972 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:59,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,972 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:53:59,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:53:59,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:59,973 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:53:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:53:59,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,974 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:53:59,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,977 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,977 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,977 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:53:59,977 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:53:59,977 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:53:59,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:53:59,977 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:53:59,977 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:53:59,978 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:53:59,978 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:53:59,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:53:59,978 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:53:59,978 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:53:59,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:53:59,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:53:59,978 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:00,079 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:00,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:00,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:00,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:00,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,087 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:00,087 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:00,087 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:54:00,087 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:00,088 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,088 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,089 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,089 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:00,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:00,090 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,090 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,090 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,091 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,091 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:54:00,091 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,091 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:54:00,091 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:00,092 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:54:00,092 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:00,092 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:00,092 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:00,092 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. started... 2023-03-10 18:54:00,092 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 01b058e52c536aa30e6d0bad142a2e3b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:00,506 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122853cd3ed5b0fa4b3f8ce61c5e3b4d14ea to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122853cd3ed5b0fa4b3f8ce61c5e3b4d14ea 2023-03-10 18:54:00,912 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/ad64a53e027e4740b453e722fec71cf6 2023-03-10 18:54:00,918 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/ad64a53e027e4740b453e722fec71cf6 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/ad64a53e027e4740b453e722fec71cf6 2023-03-10 18:54:00,924 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/ad64a53e027e4740b453e722fec71cf6, entries=1, sequenceid=21, filesize=4.9 K 2023-03-10 18:54:00,925 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 01b058e52c536aa30e6d0bad142a2e3b in 833ms, sequenceid=21, compaction requested=true 2023-03-10 18:54:00,925 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:54:00,925 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:00,925 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:00,925 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:00,925 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,925 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:54:00,925 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:54:00,927 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,927 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:00,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:00,927 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,927 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:00,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:00,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:54:00,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7dc58df6[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:00,929 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:00,929 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,930 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,930 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:00,930 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,930 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:00,930 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:54:00,930 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:00,933 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,933 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:54:00,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:00,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:00,933 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:00,933 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:54:00,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:00,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:00,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:00,936 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:00,936 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:54:00,936 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:00,936 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:00,936 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,936 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:00,940 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,940 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:00,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,940 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:00,941 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:00,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:00,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:00,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,942 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:00,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,944 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,944 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:00,944 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:00,944 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:00,944 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:00,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:00,944 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,944 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:00,944 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:00,944 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:00,945 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:00,945 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:00,945 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:00,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:00,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:01,046 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:01,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:01,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:01,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:01,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,053 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:01,053 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:01,054 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:54:01,054 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:01,054 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,054 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,057 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,057 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:01,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:01,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:01,057 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,057 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:01,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:01,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:54:01,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,058 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:54:01,058 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:01,059 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:54:01,059 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:01,059 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:01,059 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:01,059 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. started... 2023-03-10 18:54:01,059 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 01b058e52c536aa30e6d0bad142a2e3b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:01,475 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d7d8b47162c746c5aaff8ac8cc7fb103 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d7d8b47162c746c5aaff8ac8cc7fb103 2023-03-10 18:54:01,881 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/e027841d676c4037b17729c3fc9850bc 2023-03-10 18:54:01,887 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/e027841d676c4037b17729c3fc9850bc as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/e027841d676c4037b17729c3fc9850bc 2023-03-10 18:54:01,893 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/e027841d676c4037b17729c3fc9850bc, entries=1, sequenceid=25, filesize=4.9 K 2023-03-10 18:54:01,894 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 01b058e52c536aa30e6d0bad142a2e3b in 835ms, sequenceid=25, compaction requested=true 2023-03-10 18:54:01,894 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:54:01,894 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:01,894 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:01,894 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:01,894 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,894 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:54:01,894 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:54:01,896 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,896 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:01,896 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,896 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33d327f[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:01,897 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:01,897 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,899 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,899 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:01,899 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:54:01,899 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,899 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:01,899 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:01,901 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:54:01,901 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,901 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:01,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,901 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:54:01,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:01,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:01,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:01,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:01,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:01,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:01,904 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:54:01,904 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:01,904 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:01,904 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,904 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:01,907 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,907 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:01,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:01,907 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:01,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,908 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:01,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:01,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:01,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:01,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,909 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:01,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,912 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,912 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:01,912 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:01,912 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:01,912 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:01,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:54:01,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:01,912 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:01,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:01,912 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,913 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:01,913 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,913 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,913 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:01,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:01,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:01,913 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:01,913 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:02,013 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:02,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:02,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:02,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:02,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,021 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:02,021 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:02,022 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:54:02,022 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:02,022 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,022 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,024 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,024 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:02,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:02,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:02,024 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,024 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:02,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,025 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:02,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,025 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,025 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:54:02,025 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,025 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:54:02,025 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:54:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:02,026 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:02,026 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. started... 2023-03-10 18:54:02,026 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 01b058e52c536aa30e6d0bad142a2e3b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:02,442 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c2188ee22483462ebcccd9c7c6f8d616 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101c2188ee22483462ebcccd9c7c6f8d616 2023-03-10 18:54:02,847 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/4c888f9d7a67489d81c751dc4fcb3608 2023-03-10 18:54:02,853 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/4c888f9d7a67489d81c751dc4fcb3608 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/4c888f9d7a67489d81c751dc4fcb3608 2023-03-10 18:54:02,858 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/4c888f9d7a67489d81c751dc4fcb3608, entries=1, sequenceid=29, filesize=4.9 K 2023-03-10 18:54:02,859 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 01b058e52c536aa30e6d0bad142a2e3b in 833ms, sequenceid=29, compaction requested=true 2023-03-10 18:54:02,860 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:54:02,860 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:02,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:02,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:02,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:54:02,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:54:02,862 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,862 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:02,862 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,862 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:02,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:02,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:54:02,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79ded838[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:02,863 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:02,863 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,865 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,865 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:02,865 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:54:02,865 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,865 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:02,865 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:02,867 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:54:02,867 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,867 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:02,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:02,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:02,867 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:54:02,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:02,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:02,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:02,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:02,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:02,870 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:54:02,870 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:02,870 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:02,870 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,870 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:02,871 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,871 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:02,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,871 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:02,872 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,873 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:02,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,875 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:02,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:02,876 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:02,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:02,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:02,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:02,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:02,877 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,877 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,877 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,877 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:02,877 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:02,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:02,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:02,977 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:02,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:02,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:02,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:02,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,985 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:02,985 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:02,986 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:54:02,986 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:02,986 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,986 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,989 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,989 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:02,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:02,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:02,989 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:02,989 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:02,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:02,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,990 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,990 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:54:02,990 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:02,990 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:54:02,990 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:02,991 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:54:02,991 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:02,991 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:02,991 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:02,991 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. started... 2023-03-10 18:54:02,991 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 01b058e52c536aa30e6d0bad142a2e3b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:03,406 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103237f53b91d5f4178aec4f168215f2b2f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103237f53b91d5f4178aec4f168215f2b2f 2023-03-10 18:54:03,812 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/2eeccf72f86945f09daa373f32145e0a 2023-03-10 18:54:03,818 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/2eeccf72f86945f09daa373f32145e0a as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/2eeccf72f86945f09daa373f32145e0a 2023-03-10 18:54:03,822 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/2eeccf72f86945f09daa373f32145e0a, entries=1, sequenceid=33, filesize=4.9 K 2023-03-10 18:54:03,823 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 01b058e52c536aa30e6d0bad142a2e3b in 832ms, sequenceid=33, compaction requested=true 2023-03-10 18:54:03,823 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:54:03,824 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:03,824 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:03,824 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:03,824 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,824 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:54:03,824 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:54:03,825 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,825 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:03,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:03,826 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,826 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:03,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:03,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:03,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:03,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:54:03,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3efdff83[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:03,827 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:03,827 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,829 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,829 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:03,829 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:54:03,829 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:03,829 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,829 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:03,834 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:54:03,834 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:03,834 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,834 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:54:03,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:03,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:03,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:03,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:03,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:03,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:03,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:03,836 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:54:03,836 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:03,837 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:03,837 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,837 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:03,838 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,838 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:03,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:03,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,838 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:03,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:03,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:03,839 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,840 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:03,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,842 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,842 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,842 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:03,842 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:03,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:54:03,842 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:03,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:03,842 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:03,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:03,843 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:03,843 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,843 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,843 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:03,843 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:03,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:03,843 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:03,944 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:03,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:03,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:03,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:03,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,952 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:03,952 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:03,952 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:54:03,952 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:03,953 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,953 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,954 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:03,954 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:03,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:03,954 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:03,955 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:03,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:03,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,955 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:54:03,955 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:03,955 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:54:03,956 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:03,956 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:54:03,956 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:03,956 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:03,956 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:03,956 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. started... 2023-03-10 18:54:03,956 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 01b058e52c536aa30e6d0bad142a2e3b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:04,371 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509070df27d12ad094a37b3239b26604a16dd to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509070df27d12ad094a37b3239b26604a16dd 2023-03-10 18:54:04,777 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/832ee4fc4bf14e3e95f4020e22828702 2023-03-10 18:54:04,782 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/832ee4fc4bf14e3e95f4020e22828702 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/832ee4fc4bf14e3e95f4020e22828702 2023-03-10 18:54:04,788 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/832ee4fc4bf14e3e95f4020e22828702, entries=1, sequenceid=37, filesize=4.9 K 2023-03-10 18:54:04,789 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 01b058e52c536aa30e6d0bad142a2e3b in 833ms, sequenceid=37, compaction requested=true 2023-03-10 18:54:04,789 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:54:04,789 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:04,789 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:04,789 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:04,789 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,789 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:54:04,789 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:54:04,792 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,792 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:04,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:04,792 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,792 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:04,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:04,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:04,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:04,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:54:04,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f067f97[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:04,794 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:04,794 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,796 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,796 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:04,796 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:54:04,796 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,796 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:04,796 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:04,798 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:54:04,798 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:04,798 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,798 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:54:04,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:04,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:04,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:04,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:04,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:04,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:04,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:04,801 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:54:04,801 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:04,801 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:04,801 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,801 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:04,803 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,803 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:04,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:04,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,803 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:04,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:04,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:04,804 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:04,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:04,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,805 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:04,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,809 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,809 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:04,809 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:04,809 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:04,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:04,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:04,809 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:04,809 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:04,809 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:04,810 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:04,810 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,810 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:04,810 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:04,810 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,910 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:04,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:04,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:04,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:04,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,917 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:04,917 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:04,918 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:54:04,918 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:04,918 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,918 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,920 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:04,920 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:04,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:04,920 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:04,920 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:04,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:54:04,921 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:04,921 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:54:04,921 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:04,922 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:54:04,922 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:04,923 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:04,923 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:04,923 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. started... 2023-03-10 18:54:04,923 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 01b058e52c536aa30e6d0bad142a2e3b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:05,338 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112087895465de6142e694b39dc929d01877 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112087895465de6142e694b39dc929d01877 2023-03-10 18:54:05,744 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/8fd052b279eb4ef4a1524a5013b050f4 2023-03-10 18:54:05,750 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/8fd052b279eb4ef4a1524a5013b050f4 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8fd052b279eb4ef4a1524a5013b050f4 2023-03-10 18:54:05,755 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8fd052b279eb4ef4a1524a5013b050f4, entries=1, sequenceid=41, filesize=4.9 K 2023-03-10 18:54:05,756 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 01b058e52c536aa30e6d0bad142a2e3b in 833ms, sequenceid=41, compaction requested=true 2023-03-10 18:54:05,756 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:54:05,756 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:05,757 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:05,757 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:05,757 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,757 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:54:05,757 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:54:05,758 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,758 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:05,759 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,759 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:05,760 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:54:05,760 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c206c2[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:05,760 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:05,760 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,761 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,762 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:05,762 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:54:05,762 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,762 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:05,762 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:05,768 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,768 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:54:05,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:05,768 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:05,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:05,768 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:54:05,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:05,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:05,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:05,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:05,771 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:54:05,771 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:05,771 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:05,771 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,771 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:05,772 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,772 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:05,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,772 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:05,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:05,773 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:05,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:05,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:05,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:05,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,774 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:05,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,776 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,776 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,776 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:05,776 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:05,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-10 18:54:05,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:05,776 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:05,776 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:05,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:54:05,776 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:05,777 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:05,778 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,778 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:05,778 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:05,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:05,778 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:05,878 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:05,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:05,881 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-10 18:54:05,882 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:54:05,883 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:54:05,884 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:05,884 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475045884 2023-03-10 18:54:05,885 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:54:05,886 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-10 18:54:05,886 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:54:05,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x0650db47 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:54:05,891 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18f915d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:54:05,891 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:54:06,724 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035eaecbeb9e644888bfbbedcd8247f5ba to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601035eaecbeb9e644888bfbbedcd8247f5ba 2023-03-10 18:54:06,725 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:54:06,740 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:54:06,741 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37510, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:54:06,748 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:54:06,749 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48456, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:54:06,759 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3637ac67b2e2481db269140f00498770 first=Optional[r5] last=Optional[r6] 2023-03-10 18:54:06,760 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3637ac67b2e2481db269140f00498770}] 2023-03-10 18:54:06,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3637ac67b2e2481db269140f00498770 for inclusion in 01b058e52c536aa30e6d0bad142a2e3b/family1 2023-03-10 18:54:06,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-10 18:54:06,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:54:06,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:54:06,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3637ac67b2e2481db269140f00498770 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6jogcllh3os1iikpnncmen3fm91gpa52pd0thnmd588e2pltk8vtcdt3anv03sgj/family1/3637ac67b2e2481db269140f00498770 2023-03-10 18:54:06,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6jogcllh3os1iikpnncmen3fm91gpa52pd0thnmd588e2pltk8vtcdt3anv03sgj/family1/3637ac67b2e2481db269140f00498770 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/c6340f5b59cd40aca66f47d659537840_SeqId_44_ 2023-03-10 18:54:06,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6jogcllh3os1iikpnncmen3fm91gpa52pd0thnmd588e2pltk8vtcdt3anv03sgj/family1/3637ac67b2e2481db269140f00498770 into 01b058e52c536aa30e6d0bad142a2e3b/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/c6340f5b59cd40aca66f47d659537840_SeqId_44_ - updating store file list. 2023-03-10 18:54:06,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/c6340f5b59cd40aca66f47d659537840_SeqId_44_ into 01b058e52c536aa30e6d0bad142a2e3b/family1 2023-03-10 18:54:06,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6jogcllh3os1iikpnncmen3fm91gpa52pd0thnmd588e2pltk8vtcdt3anv03sgj/family1/3637ac67b2e2481db269140f00498770 into 01b058e52c536aa30e6d0bad142a2e3b/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/c6340f5b59cd40aca66f47d659537840_SeqId_44_) 2023-03-10 18:54:06,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6jogcllh3os1iikpnncmen3fm91gpa52pd0thnmd588e2pltk8vtcdt3anv03sgj/family1/3637ac67b2e2481db269140f00498770 2023-03-10 18:54:06,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-10 18:54:06,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-10 18:54:06,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): 01b058e52c536aa30e6d0bad142a2e3b/family1 is initiating minor compaction 2023-03-10 18:54:06,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7640607a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:06,780 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 01b058e52c536aa30e6d0bad142a2e3b/family1 in testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:06,780 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/5eaf0c695c48495e9e02603549a5937f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/912e37e5b3a64a5e94b3c893a4f0e538, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/54ef74abf72b418eb8b66381698164aa, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/ad64a53e027e4740b453e722fec71cf6, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/e027841d676c4037b17729c3fc9850bc, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/4c888f9d7a67489d81c751dc4fcb3608, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/2eeccf72f86945f09daa373f32145e0a, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/832ee4fc4bf14e3e95f4020e22828702, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8fd052b279eb4ef4a1524a5013b050f4, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/c6340f5b59cd40aca66f47d659537840_SeqId_44_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp, totalSize=49.7 K 2023-03-10 18:54:06,781 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 5eaf0c695c48495e9e02603549a5937f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-10 18:54:06,781 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 912e37e5b3a64a5e94b3c893a4f0e538, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-10 18:54:06,781 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 54ef74abf72b418eb8b66381698164aa, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-10 18:54:06,781 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting ad64a53e027e4740b453e722fec71cf6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-10 18:54:06,781 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting e027841d676c4037b17729c3fc9850bc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-10 18:54:06,781 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 4c888f9d7a67489d81c751dc4fcb3608, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-10 18:54:06,781 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 2eeccf72f86945f09daa373f32145e0a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-10 18:54:06,781 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 832ee4fc4bf14e3e95f4020e22828702, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-10 18:54:06,781 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 8fd052b279eb4ef4a1524a5013b050f4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-10 18:54:06,782 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting c6340f5b59cd40aca66f47d659537840_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-10 18:54:06,783 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:54:06,784 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:06,785 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101c2188ee22483462ebcccd9c7c6f8d616 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101c2188ee22483462ebcccd9c7c6f8d616 2023-03-10 18:54:06,787 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103237f53b91d5f4178aec4f168215f2b2f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103237f53b91d5f4178aec4f168215f2b2f 2023-03-10 18:54:06,787 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-10 18:54:06,787 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-10 18:54:06,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:54:06,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:54:06,832 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): 01b058e52c536aa30e6d0bad142a2e3b#family1#compaction#22 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:54:07,250 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c703d8f95dcf45acbe4a99a76e4d3f64 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231c703d8f95dcf45acbe4a99a76e4d3f64 2023-03-10 18:54:07,250 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/94e16bbca985404facd747f57d663c92 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/94e16bbca985404facd747f57d663c92 2023-03-10 18:54:07,251 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:54:07,257 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 01b058e52c536aa30e6d0bad142a2e3b/family1 of 01b058e52c536aa30e6d0bad142a2e3b into 94e16bbca985404facd747f57d663c92(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:54:07,257 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:54:07,257 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678474446780; duration=0sec 2023-03-10 18:54:07,257 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:07,277 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/104d81f5d76d4a61bcd50bd40ee20d77 first=Optional[r1] last=Optional[r4] 2023-03-10 18:54:07,278 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/104d81f5d76d4a61bcd50bd40ee20d77}] 2023-03-10 18:54:07,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/104d81f5d76d4a61bcd50bd40ee20d77 for inclusion in 01b058e52c536aa30e6d0bad142a2e3b/family1 2023-03-10 18:54:07,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-10 18:54:07,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:54:07,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:54:07,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/104d81f5d76d4a61bcd50bd40ee20d77 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__atrd9ms5eu9hqa5o7kg1tq1h4m5ahv2dt6cq0bbeabkp64j8cu8qaf0c81hb5feo/family1/104d81f5d76d4a61bcd50bd40ee20d77 2023-03-10 18:54:07,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__atrd9ms5eu9hqa5o7kg1tq1h4m5ahv2dt6cq0bbeabkp64j8cu8qaf0c81hb5feo/family1/104d81f5d76d4a61bcd50bd40ee20d77 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8858cdb529cb4032833cba2426443f98_SeqId_47_ 2023-03-10 18:54:07,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__atrd9ms5eu9hqa5o7kg1tq1h4m5ahv2dt6cq0bbeabkp64j8cu8qaf0c81hb5feo/family1/104d81f5d76d4a61bcd50bd40ee20d77 into 01b058e52c536aa30e6d0bad142a2e3b/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8858cdb529cb4032833cba2426443f98_SeqId_47_ - updating store file list. 2023-03-10 18:54:07,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8858cdb529cb4032833cba2426443f98_SeqId_47_ into 01b058e52c536aa30e6d0bad142a2e3b/family1 2023-03-10 18:54:07,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__atrd9ms5eu9hqa5o7kg1tq1h4m5ahv2dt6cq0bbeabkp64j8cu8qaf0c81hb5feo/family1/104d81f5d76d4a61bcd50bd40ee20d77 into 01b058e52c536aa30e6d0bad142a2e3b/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8858cdb529cb4032833cba2426443f98_SeqId_47_) 2023-03-10 18:54:07,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__atrd9ms5eu9hqa5o7kg1tq1h4m5ahv2dt6cq0bbeabkp64j8cu8qaf0c81hb5feo/family1/104d81f5d76d4a61bcd50bd40ee20d77 2023-03-10 18:54:07,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-10 18:54:07,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-10 18:54:07,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): 01b058e52c536aa30e6d0bad142a2e3b/family1 is initiating minor compaction (all files) 2023-03-10 18:54:07,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@68f563f8; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:07,294 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 01b058e52c536aa30e6d0bad142a2e3b/family1 in testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:54:07,294 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/6832806035aa42bf9fa14e3c8ae502b8, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/94e16bbca985404facd747f57d663c92, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8858cdb529cb4032833cba2426443f98_SeqId_47_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp, totalSize=178.1 K 2023-03-10 18:54:07,295 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 6832806035aa42bf9fa14e3c8ae502b8, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839360 2023-03-10 18:54:07,295 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 94e16bbca985404facd747f57d663c92, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239360 2023-03-10 18:54:07,296 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 8858cdb529cb4032833cba2426443f98_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965239360 2023-03-10 18:54:07,296 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:54:07,297 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:07,299 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201c5096d21eb964cae9501ec07dea2fbc4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201c5096d21eb964cae9501ec07dea2fbc4 2023-03-10 18:54:07,300 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f8a72ce024da48f184fb71d7d2f9c75a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f8a72ce024da48f184fb71d7d2f9c75a 2023-03-10 18:54:07,301 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122853cd3ed5b0fa4b3f8ce61c5e3b4d14ea to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122853cd3ed5b0fa4b3f8ce61c5e3b4d14ea 2023-03-10 18:54:07,303 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d7d8b47162c746c5aaff8ac8cc7fb103 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d7d8b47162c746c5aaff8ac8cc7fb103 2023-03-10 18:54:07,303 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-10 18:54:07,303 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-10 18:54:07,304 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:54:07,304 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:54:07,317 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): 01b058e52c536aa30e6d0bad142a2e3b#family1#compaction#23 average throughput is 11.97 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:54:07,330 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/5eaf0c695c48495e9e02603549a5937f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/912e37e5b3a64a5e94b3c893a4f0e538, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/54ef74abf72b418eb8b66381698164aa, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/ad64a53e027e4740b453e722fec71cf6, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/e027841d676c4037b17729c3fc9850bc, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/4c888f9d7a67489d81c751dc4fcb3608, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/2eeccf72f86945f09daa373f32145e0a, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/832ee4fc4bf14e3e95f4020e22828702, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8fd052b279eb4ef4a1524a5013b050f4, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/c6340f5b59cd40aca66f47d659537840_SeqId_44_] to archive 2023-03-10 18:54:07,331 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:07,334 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/5eaf0c695c48495e9e02603549a5937f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/5eaf0c695c48495e9e02603549a5937f 2023-03-10 18:54:07,336 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/912e37e5b3a64a5e94b3c893a4f0e538 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/912e37e5b3a64a5e94b3c893a4f0e538 2023-03-10 18:54:07,337 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/54ef74abf72b418eb8b66381698164aa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/54ef74abf72b418eb8b66381698164aa 2023-03-10 18:54:07,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/ad64a53e027e4740b453e722fec71cf6 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/ad64a53e027e4740b453e722fec71cf6 2023-03-10 18:54:07,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/e027841d676c4037b17729c3fc9850bc to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/e027841d676c4037b17729c3fc9850bc 2023-03-10 18:54:07,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/4c888f9d7a67489d81c751dc4fcb3608 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/4c888f9d7a67489d81c751dc4fcb3608 2023-03-10 18:54:07,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/2eeccf72f86945f09daa373f32145e0a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/2eeccf72f86945f09daa373f32145e0a 2023-03-10 18:54:07,345 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/832ee4fc4bf14e3e95f4020e22828702 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/832ee4fc4bf14e3e95f4020e22828702 2023-03-10 18:54:07,346 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8fd052b279eb4ef4a1524a5013b050f4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8fd052b279eb4ef4a1524a5013b050f4 2023-03-10 18:54:07,348 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/c6340f5b59cd40aca66f47d659537840_SeqId_44_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/c6340f5b59cd40aca66f47d659537840_SeqId_44_ 2023-03-10 18:54:07,769 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130b392ff9e329343c5b3ded2b0094391a9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130b392ff9e329343c5b3ded2b0094391a9 2023-03-10 18:54:07,770 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:54:07,797 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/186c57580403417f814ace97d01376e1 first=Optional[r00] last=Optional[r8] 2023-03-10 18:54:07,798 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/186c57580403417f814ace97d01376e1}] 2023-03-10 18:54:07,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/186c57580403417f814ace97d01376e1 for inclusion in 01b058e52c536aa30e6d0bad142a2e3b/family1 2023-03-10 18:54:07,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-10 18:54:07,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:54:07,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:54:07,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/186c57580403417f814ace97d01376e1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__kql3ipe7gnktq3vfis92r37tltanfo4opoistjvdn9bspbvjdmd8qigrocp96usd/family1/186c57580403417f814ace97d01376e1 2023-03-10 18:54:07,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__kql3ipe7gnktq3vfis92r37tltanfo4opoistjvdn9bspbvjdmd8qigrocp96usd/family1/186c57580403417f814ace97d01376e1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/f086cb2aa9674fd7a9796e5688c69c7e_SeqId_49_ 2023-03-10 18:54:07,808 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__kql3ipe7gnktq3vfis92r37tltanfo4opoistjvdn9bspbvjdmd8qigrocp96usd/family1/186c57580403417f814ace97d01376e1 into 01b058e52c536aa30e6d0bad142a2e3b/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/f086cb2aa9674fd7a9796e5688c69c7e_SeqId_49_ - updating store file list. 2023-03-10 18:54:07,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/f086cb2aa9674fd7a9796e5688c69c7e_SeqId_49_ into 01b058e52c536aa30e6d0bad142a2e3b/family1 2023-03-10 18:54:07,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__kql3ipe7gnktq3vfis92r37tltanfo4opoistjvdn9bspbvjdmd8qigrocp96usd/family1/186c57580403417f814ace97d01376e1 into 01b058e52c536aa30e6d0bad142a2e3b/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/f086cb2aa9674fd7a9796e5688c69c7e_SeqId_49_) 2023-03-10 18:54:07,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithMonthlyPolicy__kql3ipe7gnktq3vfis92r37tltanfo4opoistjvdn9bspbvjdmd8qigrocp96usd/family1/186c57580403417f814ace97d01376e1 2023-03-10 18:54:07,816 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:54:07,816 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:07,818 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112087895465de6142e694b39dc929d01877 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112087895465de6142e694b39dc929d01877 2023-03-10 18:54:07,819 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128a2deb52f0c474882987f8fafe18adbe3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128a2deb52f0c474882987f8fafe18adbe3 2023-03-10 18:54:07,821 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511307bb86a9570c848a2be35c063da1c73d1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511307bb86a9570c848a2be35c063da1c73d1 2023-03-10 18:54:07,821 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-10 18:54:07,821 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-10 18:54:07,821 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:54:07,821 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:54:07,822 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:54:07,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:54:07,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:07,823 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:54:07,825 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9420 sec 2023-03-10 18:54:07,931 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=264 (was 248) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48270 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48180 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48266 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48302 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-922730382_15 at /127.0.0.1:48336 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-922730382_15 at /127.0.0.1:48396 [Receiving block BP-875478649-172.31.14.131-1678474410466:blk_1073741898_1074] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48198 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48352 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48366 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48272 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x34b5fe46-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x34b5fe46-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-922730382_15 at /127.0.0.1:48124 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48250 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x34b5fe46-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48210 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48286 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48154 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x34b5fe46-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48428 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x34b5fe46-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48322 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56431@0x0650db47-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x34b5fe46-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123127f14ccd940247029391369ff065b24d java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48312 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48294 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56431@0x0650db47-SendThread(127.0.0.1:56431) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ResponseProcessor for block BP-875478649-172.31.14.131-1678474410466:blk_1073741898_1074 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: hconnection-0x34b5fe46-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56431@0x0650db47 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/229095779.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/d4cae74c444045ce83524b4a14637bb5 block BP-875478649-172.31.14.131-1678474410466:blk_1073741898_1074 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: PacketResponder: BP-875478649-172.31.14.131-1678474410466:blk_1073741898_1074, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_60461855_15 at /127.0.0.1:48296 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4fd27954-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=624 (was 592) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=260 (was 281), ProcessCount=170 (was 170), AvailableMemoryMB=6136 (was 5616) - AvailableMemoryMB LEAK? - 2023-03-10 18:54:07,944 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=263, OpenFileDescriptor=624, MaxFileDescriptor=60000, SystemLoadAverage=260, ProcessCount=170, AvailableMemoryMB=6135 2023-03-10 18:54:07,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-10 18:54:07,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:07,949 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:54:07,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-10 18:54:07,950 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:54:07,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-10 18:54:07,952 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b 2023-03-10 18:54:07,953 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b empty. 2023-03-10 18:54:07,953 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b 2023-03-10 18:54:07,953 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-10 18:54:08,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-10 18:54:08,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-10 18:54:08,364 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-10 18:54:08,366 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => aaa7ef415c2e1954be313179d143a66b, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:54:08,373 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:08,373 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing aaa7ef415c2e1954be313179d143a66b, disabling compactions & flushes 2023-03-10 18:54:08,373 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:08,373 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:08,373 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. after waiting 0 ms 2023-03-10 18:54:08,373 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:08,373 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:08,373 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:08,376 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:54:08,377 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678474448376"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474448376"}]},"ts":"1678474448376"} 2023-03-10 18:54:08,378 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-10 18:54:08,379 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:54:08,379 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474448379"}]},"ts":"1678474448379"} 2023-03-10 18:54:08,380 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-10 18:54:08,387 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=aaa7ef415c2e1954be313179d143a66b, ASSIGN}] 2023-03-10 18:54:08,389 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=aaa7ef415c2e1954be313179d143a66b, ASSIGN 2023-03-10 18:54:08,390 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=aaa7ef415c2e1954be313179d143a66b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:54:08,538 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/.tmp/family1/d4cae74c444045ce83524b4a14637bb5 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/d4cae74c444045ce83524b4a14637bb5 2023-03-10 18:54:08,541 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=aaa7ef415c2e1954be313179d143a66b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:08,541 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678474448541"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474448541"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474448541"}]},"ts":"1678474448541"} 2023-03-10 18:54:08,543 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure aaa7ef415c2e1954be313179d143a66b, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:54:08,545 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 01b058e52c536aa30e6d0bad142a2e3b/family1 of 01b058e52c536aa30e6d0bad142a2e3b into d4cae74c444045ce83524b4a14637bb5(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:54:08,545 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:54:08,545 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678474447294; duration=1sec 2023-03-10 18:54:08,545 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:08,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-10 18:54:08,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/6832806035aa42bf9fa14e3c8ae502b8, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/94e16bbca985404facd747f57d663c92, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8858cdb529cb4032833cba2426443f98_SeqId_47_] to archive 2023-03-10 18:54:08,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:08,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/6832806035aa42bf9fa14e3c8ae502b8 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/6832806035aa42bf9fa14e3c8ae502b8 2023-03-10 18:54:08,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/94e16bbca985404facd747f57d663c92 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/94e16bbca985404facd747f57d663c92 2023-03-10 18:54:08,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8858cdb529cb4032833cba2426443f98_SeqId_47_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/family1/8858cdb529cb4032833cba2426443f98_SeqId_47_ 2023-03-10 18:54:08,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:08,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => aaa7ef415c2e1954be313179d143a66b, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:54:08,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy aaa7ef415c2e1954be313179d143a66b 2023-03-10 18:54:08,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:08,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for aaa7ef415c2e1954be313179d143a66b 2023-03-10 18:54:08,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for aaa7ef415c2e1954be313179d143a66b 2023-03-10 18:54:08,700 INFO [StoreOpener-aaa7ef415c2e1954be313179d143a66b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region aaa7ef415c2e1954be313179d143a66b 2023-03-10 18:54:08,701 DEBUG [StoreOpener-aaa7ef415c2e1954be313179d143a66b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1 2023-03-10 18:54:08,701 DEBUG [StoreOpener-aaa7ef415c2e1954be313179d143a66b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1 2023-03-10 18:54:08,702 INFO [StoreOpener-aaa7ef415c2e1954be313179d143a66b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aaa7ef415c2e1954be313179d143a66b columnFamilyName family1 2023-03-10 18:54:08,703 INFO [StoreOpener-aaa7ef415c2e1954be313179d143a66b-1] regionserver.HStore(310): Store=aaa7ef415c2e1954be313179d143a66b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:08,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b 2023-03-10 18:54:08,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b 2023-03-10 18:54:08,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for aaa7ef415c2e1954be313179d143a66b 2023-03-10 18:54:08,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:54:08,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened aaa7ef415c2e1954be313179d143a66b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72862528, jitterRate=0.08573627471923828}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:54:08,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:08,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b., pid=25, masterSystemTime=1678474448695 2023-03-10 18:54:08,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:08,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:08,712 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=aaa7ef415c2e1954be313179d143a66b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:08,712 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678474448712"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474448712"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474448712"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474448712"}]},"ts":"1678474448712"} 2023-03-10 18:54:08,716 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-10 18:54:08,716 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure aaa7ef415c2e1954be313179d143a66b, server=jenkins-hbase4.apache.org,43181,1678474414959 in 171 msec 2023-03-10 18:54:08,717 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-10 18:54:08,717 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=aaa7ef415c2e1954be313179d143a66b, ASSIGN in 329 msec 2023-03-10 18:54:08,718 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:54:08,718 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474448718"}]},"ts":"1678474448718"} 2023-03-10 18:54:08,719 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-10 18:54:08,722 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:54:08,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 776 msec 2023-03-10 18:54:09,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-10 18:54:09,055 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-10 18:54:09,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:09,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,079 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:09,080 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:09,080 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:09,080 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:09,080 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,080 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,081 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:09,081 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:09,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:09,082 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,082 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:09,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,082 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:09,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,083 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,083 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,083 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:09,083 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:09,083 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:09,083 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:09,083 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:09,083 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:09,083 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. started... 2023-03-10 18:54:09,084 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aaa7ef415c2e1954be313179d143a66b 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-10 18:54:09,507 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113042759703e0b14092bf7d9fe2aa280165 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015113042759703e0b14092bf7d9fe2aa280165 2023-03-10 18:54:09,911 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/527882d24a1d414f92c4a635856c872c 2023-03-10 18:54:09,917 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/527882d24a1d414f92c4a635856c872c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/527882d24a1d414f92c4a635856c872c 2023-03-10 18:54:09,941 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:54:09,941 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x67868145 to 127.0.0.1:56431 2023-03-10 18:54:09,941 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:54:09,942 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/527882d24a1d414f92c4a635856c872c, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-10 18:54:09,942 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:54:09,942 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x38a3fd56 to 127.0.0.1:56431 2023-03-10 18:54:09,942 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:54:09,943 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:54:09,943 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0650db47 to 127.0.0.1:56431 2023-03-10 18:54:09,943 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:54:09,943 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for aaa7ef415c2e1954be313179d143a66b in 859ms, sequenceid=5, compaction requested=false 2023-03-10 18:54:09,944 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:09,944 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:09,945 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:09,945 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:09,945 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:09,945 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,945 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:09,945 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:09,947 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,947 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:09,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:09,947 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,947 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:09,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:09,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:09,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:09,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:09,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ad2115a[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:09,949 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:09,949 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,950 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,950 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:09,950 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:09,950 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,950 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:09,950 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:09,954 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:09,954 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,954 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:09,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,954 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:09,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:09,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:09,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:09,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:09,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:09,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:09,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:09,956 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:09,956 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:09,956 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:09,956 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,956 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:09,958 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,958 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,958 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:09,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:09,958 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:09,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,959 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:09,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,962 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,962 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:09,962 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,962 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:09,962 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:09,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:09,962 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:09,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:09,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:09,962 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:09,962 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,962 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,962 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:09,962 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:09,962 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:09,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:09,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:10,063 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:10,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:10,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:10,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:10,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,070 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:10,070 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:10,070 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:10,070 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:10,070 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,070 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,072 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,072 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:10,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:10,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:10,072 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,072 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:10,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:10,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,073 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,073 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:10,073 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,073 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:10,073 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:10,073 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:10,073 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:10,073 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:10,073 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:10,073 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. started... 2023-03-10 18:54:10,074 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aaa7ef415c2e1954be313179d143a66b 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-10 18:54:10,487 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112817d9ca8ad6dd42bfb90e4ad3e6f58faf to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015112817d9ca8ad6dd42bfb90e4ad3e6f58faf 2023-03-10 18:54:10,892 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/7d18e7e493d3428db94ba6ae2b8072f2 2023-03-10 18:54:10,898 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/7d18e7e493d3428db94ba6ae2b8072f2 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7d18e7e493d3428db94ba6ae2b8072f2 2023-03-10 18:54:10,902 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7d18e7e493d3428db94ba6ae2b8072f2, entries=1, sequenceid=9, filesize=5.0 K 2023-03-10 18:54:10,903 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for aaa7ef415c2e1954be313179d143a66b in 829ms, sequenceid=9, compaction requested=false 2023-03-10 18:54:10,903 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:10,903 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:10,903 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:10,903 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:10,903 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,903 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:10,903 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:10,906 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,906 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:10,906 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,907 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:10,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:10,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:10,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:10,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:10,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b82e664[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:10,908 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:10,908 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,909 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,909 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:10,909 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:10,909 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,909 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:10,909 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:10,911 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:10,911 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,911 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:10,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:10,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:10,911 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:10,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:10,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:10,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:10,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:10,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:10,913 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:10,913 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:10,913 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:10,913 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,913 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:10,914 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,914 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,914 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:10,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:10,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:10,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:10,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:10,915 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:10,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:10,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,916 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:10,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,924 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,924 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,924 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,924 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:10,924 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:10,924 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:10,924 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:10,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:10,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-10 18:54:10,924 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:10,924 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:10,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:10,925 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:10,925 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:10,925 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:10,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:10,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:11,025 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:11,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:11,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:11,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:11,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,032 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:11,032 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:11,032 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:11,032 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:11,032 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,032 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,034 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,034 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:11,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:11,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:11,034 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,034 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:11,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:11,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,035 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,035 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:11,035 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,035 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:11,035 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:11,035 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:11,035 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:11,035 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:11,035 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:11,035 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. started... 2023-03-10 18:54:11,036 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aaa7ef415c2e1954be313179d143a66b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:11,449 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201c927d871771c4438bc26b2427a7d3c78 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201c927d871771c4438bc26b2427a7d3c78 2023-03-10 18:54:11,454 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/37211e5a3b584d47ac2d3c42d211307d 2023-03-10 18:54:11,459 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/37211e5a3b584d47ac2d3c42d211307d as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/37211e5a3b584d47ac2d3c42d211307d 2023-03-10 18:54:11,464 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/37211e5a3b584d47ac2d3c42d211307d, entries=1, sequenceid=13, filesize=4.9 K 2023-03-10 18:54:11,465 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aaa7ef415c2e1954be313179d143a66b in 429ms, sequenceid=13, compaction requested=true 2023-03-10 18:54:11,465 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:11,465 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:11,465 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:11,465 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:11,465 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,465 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:11,465 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:11,467 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,467 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:11,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:11,468 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,468 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:11,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:11,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:11,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:11,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:11,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a4c044[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:11,469 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:11,469 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,470 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,471 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:11,471 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:11,471 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:11,471 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,471 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:11,473 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,473 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:11,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:11,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:11,473 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:11,473 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:11,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:11,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:11,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:11,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:11,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:11,475 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:11,475 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:11,476 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:11,476 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,476 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:11,479 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,479 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:11,479 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:11,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:11,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:11,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:11,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:11,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,480 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:11,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,481 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:11,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,484 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:11,484 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:11,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:11,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:11,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:11,485 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:11,485 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,485 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,485 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,485 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,485 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,485 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,485 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:11,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:11,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:11,485 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:11,485 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:11,585 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:11,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:11,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:11,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:11,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,595 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:11,595 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:11,595 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:11,595 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:11,596 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,596 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,598 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,598 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:11,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:11,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:11,599 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:11,599 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:11,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,599 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:11,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,599 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,600 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:11,600 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:11,600 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:11,600 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:11,600 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:11,600 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:11,600 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:11,600 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:11,600 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. started... 2023-03-10 18:54:11,600 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aaa7ef415c2e1954be313179d143a66b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:12,016 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205627ddcadff5e422f9914d11c2aa5f952 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205627ddcadff5e422f9914d11c2aa5f952 2023-03-10 18:54:12,422 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/3b9235826e8445889156cf618d00352d 2023-03-10 18:54:12,428 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/3b9235826e8445889156cf618d00352d as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3b9235826e8445889156cf618d00352d 2023-03-10 18:54:12,432 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3b9235826e8445889156cf618d00352d, entries=1, sequenceid=17, filesize=4.9 K 2023-03-10 18:54:12,433 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aaa7ef415c2e1954be313179d143a66b in 833ms, sequenceid=17, compaction requested=true 2023-03-10 18:54:12,434 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:12,434 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:12,434 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:12,434 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:12,434 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,434 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:12,434 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:12,436 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,436 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:12,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:12,436 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,436 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:12,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:12,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:12,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:12,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:12,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12c52264[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:12,437 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:12,437 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,438 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,439 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:12,439 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:12,439 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:12,439 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,439 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:12,440 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,440 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:12,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,440 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:12,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:12,440 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:12,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:12,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:12,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:12,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:12,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:12,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:12,442 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:12,442 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:12,442 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:12,442 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,442 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:12,445 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,445 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,445 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:12,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:12,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:12,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,445 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:12,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:12,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:12,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:12,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,446 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:12,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,449 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,449 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,449 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:12,449 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,449 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:12,449 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:12,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:12,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:12,449 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:12,449 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:12,449 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,449 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,449 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:12,449 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:12,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:12,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:12,550 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:12,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:12,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:12,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:12,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,560 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:12,560 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:12,560 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:12,560 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:12,560 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,560 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,562 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,562 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:12,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:12,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:12,563 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:12,563 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:12,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,563 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:12,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,564 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:12,564 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:12,564 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:12,564 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:12,564 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:12,565 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:12,565 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:12,565 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:12,565 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. started... 2023-03-10 18:54:12,565 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aaa7ef415c2e1954be313179d143a66b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:12,984 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d67a27e2c80547d5b8870adbfe5978f4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d67a27e2c80547d5b8870adbfe5978f4 2023-03-10 18:54:13,393 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/0c0c6c993c8d432090bc2371a9b7b473 2023-03-10 18:54:13,399 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/0c0c6c993c8d432090bc2371a9b7b473 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/0c0c6c993c8d432090bc2371a9b7b473 2023-03-10 18:54:13,405 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/0c0c6c993c8d432090bc2371a9b7b473, entries=1, sequenceid=21, filesize=4.9 K 2023-03-10 18:54:13,406 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aaa7ef415c2e1954be313179d143a66b in 841ms, sequenceid=21, compaction requested=true 2023-03-10 18:54:13,406 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:13,406 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:13,406 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:13,406 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:13,406 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,407 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:13,407 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:13,409 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,409 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:13,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:13,409 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:13,409 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:13,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:13,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:13,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:13,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@887a9a7[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:13,411 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:13,411 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,417 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,418 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:13,418 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:13,418 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:13,418 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,418 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:13,424 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:13,425 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,425 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:13,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:13,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:13,425 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:13,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:13,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:13,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:13,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:13,427 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:13,427 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:13,427 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:13,427 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:13,427 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,427 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:13,433 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,433 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:13,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,433 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:13,434 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:13,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,436 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:13,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,440 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:13,440 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:13,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:13,440 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,440 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:13,440 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:13,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:13,440 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,440 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:13,440 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:13,440 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,441 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:13,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:13,441 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:13,441 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,541 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:13,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:13,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:13,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:13,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,549 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:13,549 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:13,549 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:13,549 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:13,549 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,549 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,555 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,555 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:13,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:13,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:13,555 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,556 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:13,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:13,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,556 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,557 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:13,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,557 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:13,557 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:13,557 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:13,557 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:13,557 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:13,557 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:13,557 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. started... 2023-03-10 18:54:13,557 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aaa7ef415c2e1954be313179d143a66b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:13,575 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512310e7837f323a744f49836f2a5a2a531ea to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512310e7837f323a744f49836f2a5a2a531ea 2023-03-10 18:54:13,586 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/5e495c254a7f497398d4fd4fb79182c4 2023-03-10 18:54:13,591 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/5e495c254a7f497398d4fd4fb79182c4 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5e495c254a7f497398d4fd4fb79182c4 2023-03-10 18:54:13,598 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5e495c254a7f497398d4fd4fb79182c4, entries=1, sequenceid=25, filesize=4.9 K 2023-03-10 18:54:13,599 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aaa7ef415c2e1954be313179d143a66b in 42ms, sequenceid=25, compaction requested=true 2023-03-10 18:54:13,599 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:13,599 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:13,599 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:13,599 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:13,599 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,599 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:13,599 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:13,601 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,601 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:13,601 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,601 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:13,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:13,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44facf15[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:13,603 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:13,603 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,604 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,604 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:13,605 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:13,605 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:13,610 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:13,610 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:13,610 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,611 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:13,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:13,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:13,611 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:13,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:13,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:13,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,611 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:13,613 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,613 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:13,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,613 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:13,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:13,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:13,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:13,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:13,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,614 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:13,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:13,615 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,619 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,619 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,619 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,619 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:13,619 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:13,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:13,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:13,619 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,620 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,620 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,620 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,620 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:13,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:13,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:13,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:13,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:13,620 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:13,620 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:13,721 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:13,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:13,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:13,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:13,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,733 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:13,733 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:13,733 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:13,733 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:13,733 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,733 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,735 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:13,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:13,735 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:13,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,736 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:13,736 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:13,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:13,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,737 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:13,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:13,737 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:13,737 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:13,737 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:13,737 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:13,737 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:13,738 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:13,738 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. started... 2023-03-10 18:54:13,738 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aaa7ef415c2e1954be313179d143a66b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:14,164 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601018b59b1b679b843c1a64861f13e618925 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601018b59b1b679b843c1a64861f13e618925 2023-03-10 18:54:14,170 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/5d73bf257291417a838966121afa7c6d 2023-03-10 18:54:14,177 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/5d73bf257291417a838966121afa7c6d as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5d73bf257291417a838966121afa7c6d 2023-03-10 18:54:14,182 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5d73bf257291417a838966121afa7c6d, entries=1, sequenceid=29, filesize=4.9 K 2023-03-10 18:54:14,183 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aaa7ef415c2e1954be313179d143a66b in 445ms, sequenceid=29, compaction requested=true 2023-03-10 18:54:14,183 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:14,183 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:14,183 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:14,183 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:14,183 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,184 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:14,184 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:14,186 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,186 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:14,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:14,186 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,186 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:14,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:14,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:14,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:14,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:14,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@657d812e[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:14,188 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:14,188 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,190 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,190 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:14,190 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:14,190 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,190 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:14,190 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:14,192 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:14,192 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:14,192 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,192 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:14,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:14,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:14,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:14,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:14,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:14,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:14,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:14,195 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:14,195 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:14,195 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:14,195 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,195 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:14,197 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,197 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:14,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:14,197 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:14,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:14,198 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:14,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:14,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:14,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,199 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:14,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,202 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,202 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,202 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,202 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,202 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:14,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:14,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:14,202 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:14,202 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,202 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,203 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,202 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:14,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:14,203 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:14,203 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:14,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:14,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:14,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:14,238 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:54:14,303 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:14,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:14,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:14,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:14,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,312 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:14,312 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:14,312 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:14,312 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:14,312 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,312 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,315 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:14,315 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:14,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:14,315 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:14,316 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:14,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,316 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:14,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,317 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,317 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:14,317 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:14,317 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:14,317 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:14,317 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:14,317 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:14,317 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:14,317 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:14,318 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. started... 2023-03-10 18:54:14,318 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aaa7ef415c2e1954be313179d143a66b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:14,744 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103db044b05c92c4eaf92d7c0314b4981e8 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103db044b05c92c4eaf92d7c0314b4981e8 2023-03-10 18:54:15,151 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/7daade243ded449c90702da866a76a59 2023-03-10 18:54:15,157 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/7daade243ded449c90702da866a76a59 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7daade243ded449c90702da866a76a59 2023-03-10 18:54:15,162 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7daade243ded449c90702da866a76a59, entries=1, sequenceid=33, filesize=4.9 K 2023-03-10 18:54:15,164 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aaa7ef415c2e1954be313179d143a66b in 846ms, sequenceid=33, compaction requested=true 2023-03-10 18:54:15,164 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:15,164 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:15,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:15,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:15,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:15,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:15,166 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,166 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:15,166 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,166 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:15,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:15,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:15,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:15,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e1059e7[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:15,168 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:15,168 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,169 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:15,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:15,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:15,170 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,170 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:15,171 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:15,171 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,171 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:15,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:15,172 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:15,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:15,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:15,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:15,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:15,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:15,174 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:15,175 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:15,175 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:15,175 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,175 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:15,177 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,177 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:15,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:15,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,178 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:15,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:15,178 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:15,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:15,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,179 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:15,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,181 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,181 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:15,181 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:15,182 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:15,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:15,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:15,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-10 18:54:15,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:15,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:15,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,182 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:15,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,182 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:15,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:15,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:15,282 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:15,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:15,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:15,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:15,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,289 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:15,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:15,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:15,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:15,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,292 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,292 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:15,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:15,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:15,292 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:15,292 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:15,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:15,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:15,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:15,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:15,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:15,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:15,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:15,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:15,293 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:15,293 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. started... 2023-03-10 18:54:15,294 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aaa7ef415c2e1954be313179d143a66b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:15,707 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509072e756cda71fc442bb061d0579252d69f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201509072e756cda71fc442bb061d0579252d69f 2023-03-10 18:54:16,113 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/2fc8ad69dbe3436abd9f185a60adc34b 2023-03-10 18:54:16,118 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/2fc8ad69dbe3436abd9f185a60adc34b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/2fc8ad69dbe3436abd9f185a60adc34b 2023-03-10 18:54:16,123 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/2fc8ad69dbe3436abd9f185a60adc34b, entries=1, sequenceid=37, filesize=4.9 K 2023-03-10 18:54:16,124 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aaa7ef415c2e1954be313179d143a66b in 830ms, sequenceid=37, compaction requested=true 2023-03-10 18:54:16,124 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:16,124 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:16,124 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:16,124 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:16,124 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,124 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:16,124 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:16,126 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,126 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:16,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:16,126 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,126 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:16,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:16,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:16,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:16,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:16,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b6f1913[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:16,128 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:16,128 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,136 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,136 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:16,136 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:16,136 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:16,136 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,136 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:16,138 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:16,138 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:16,138 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,138 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:16,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:16,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:16,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:16,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:16,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:16,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:16,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:16,140 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:16,140 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:16,140 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:16,140 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,140 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:16,146 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,146 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:16,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:16,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,146 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:16,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:16,146 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:16,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:16,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:16,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,147 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:16,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,150 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,150 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:16,150 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:16,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:16,150 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:16,150 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:16,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:16,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:16,150 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,150 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,151 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,151 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,151 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:16,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:16,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-10 18:54:16,151 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:16,151 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:16,251 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:16,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:16,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:16,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:16,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,258 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:16,258 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:16,258 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:16,258 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:16,259 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,259 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,261 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,261 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:16,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:16,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:16,262 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,262 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:16,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,262 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:16,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,263 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,263 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:16,263 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,263 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:16,263 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:16,263 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:16,263 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:16,263 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:16,263 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:16,264 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. started... 2023-03-10 18:54:16,264 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aaa7ef415c2e1954be313179d143a66b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:16,278 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120d9cde1228c784133a3b2be893d4e96cf to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120d9cde1228c784133a3b2be893d4e96cf 2023-03-10 18:54:16,688 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/daf3147278d8400fa77df3a9eb75cd5d 2023-03-10 18:54:16,693 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/daf3147278d8400fa77df3a9eb75cd5d as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/daf3147278d8400fa77df3a9eb75cd5d 2023-03-10 18:54:16,698 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/daf3147278d8400fa77df3a9eb75cd5d, entries=1, sequenceid=41, filesize=4.9 K 2023-03-10 18:54:16,699 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aaa7ef415c2e1954be313179d143a66b in 435ms, sequenceid=41, compaction requested=true 2023-03-10 18:54:16,700 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:16,700 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:16,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:16,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:16,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:16,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:16,702 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,702 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:16,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:16,702 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,702 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:16,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:16,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:16,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:16,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:16,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@797ff10c[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:16,704 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:16,704 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,705 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,705 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:16,706 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:16,706 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,706 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:16,706 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:16,707 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,707 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:16,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:16,707 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:16,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:16,707 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:16,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:16,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:16,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:16,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:16,710 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:16,710 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:16,710 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:16,710 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:16,710 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,710 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:16,713 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,713 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:16,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:16,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,713 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:16,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:16,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:16,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:16,713 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:16,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,715 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:16,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,717 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,717 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:16,717 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,717 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:16,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-10 18:54:16,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:16,717 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,717 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:16,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:16,717 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:16,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:16,718 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,718 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,718 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,718 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:16,718 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:16,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:16,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:16,818 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:16,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:16,821 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-10 18:54:16,823 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:54:16,823 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:54:16,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:16,824 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475056824 2023-03-10 18:54:16,825 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-10 18:54:16,826 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-10 18:54:16,827 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:16,827 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x56649942 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:54:16,831 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b1cffaa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:54:16,832 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:17,678 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ca26b6878190403b8598f85cf4ed2e37 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ca26b6878190403b8598f85cf4ed2e37 2023-03-10 18:54:17,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:17,692 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:54:17,693 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46046, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:54:17,700 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:54:17,702 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49488, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:54:17,713 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/940eb57662d84924a1a73a86c0723827 first=Optional[r3] last=Optional[r6] 2023-03-10 18:54:17,713 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/940eb57662d84924a1a73a86c0723827}] 2023-03-10 18:54:17,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/940eb57662d84924a1a73a86c0723827 for inclusion in aaa7ef415c2e1954be313179d143a66b/family1 2023-03-10 18:54:17,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-10 18:54:17,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:54:17,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:17,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/940eb57662d84924a1a73a86c0723827 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v1dnm5vt9m9qksm5hn8dgabs38lf35udrtcri8kc3koo3rplgjuqdq1m1g10t4ad/family1/940eb57662d84924a1a73a86c0723827 2023-03-10 18:54:17,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v1dnm5vt9m9qksm5hn8dgabs38lf35udrtcri8kc3koo3rplgjuqdq1m1g10t4ad/family1/940eb57662d84924a1a73a86c0723827 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3e049f2c08d7462abfd74798d58fe2ee_SeqId_44_ 2023-03-10 18:54:17,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v1dnm5vt9m9qksm5hn8dgabs38lf35udrtcri8kc3koo3rplgjuqdq1m1g10t4ad/family1/940eb57662d84924a1a73a86c0723827 into aaa7ef415c2e1954be313179d143a66b/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3e049f2c08d7462abfd74798d58fe2ee_SeqId_44_ - updating store file list. 2023-03-10 18:54:17,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3e049f2c08d7462abfd74798d58fe2ee_SeqId_44_ into aaa7ef415c2e1954be313179d143a66b/family1 2023-03-10 18:54:17,731 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v1dnm5vt9m9qksm5hn8dgabs38lf35udrtcri8kc3koo3rplgjuqdq1m1g10t4ad/family1/940eb57662d84924a1a73a86c0723827 into aaa7ef415c2e1954be313179d143a66b/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3e049f2c08d7462abfd74798d58fe2ee_SeqId_44_) 2023-03-10 18:54:17,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v1dnm5vt9m9qksm5hn8dgabs38lf35udrtcri8kc3koo3rplgjuqdq1m1g10t4ad/family1/940eb57662d84924a1a73a86c0723827 2023-03-10 18:54:17,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-10 18:54:17,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-10 18:54:17,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): aaa7ef415c2e1954be313179d143a66b/family1 is initiating minor compaction 2023-03-10 18:54:17,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1dbd5bb6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:17,736 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of aaa7ef415c2e1954be313179d143a66b/family1 in testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:17,736 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7d18e7e493d3428db94ba6ae2b8072f2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/37211e5a3b584d47ac2d3c42d211307d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3b9235826e8445889156cf618d00352d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/0c0c6c993c8d432090bc2371a9b7b473, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5e495c254a7f497398d4fd4fb79182c4, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5d73bf257291417a838966121afa7c6d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7daade243ded449c90702da866a76a59, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/2fc8ad69dbe3436abd9f185a60adc34b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/daf3147278d8400fa77df3a9eb75cd5d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3e049f2c08d7462abfd74798d58fe2ee_SeqId_44_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp, totalSize=50.0 K 2023-03-10 18:54:17,736 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 7d18e7e493d3428db94ba6ae2b8072f2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-10 18:54:17,736 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 37211e5a3b584d47ac2d3c42d211307d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-10 18:54:17,736 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 3b9235826e8445889156cf618d00352d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-10 18:54:17,736 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 0c0c6c993c8d432090bc2371a9b7b473, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-10 18:54:17,737 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 5e495c254a7f497398d4fd4fb79182c4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-10 18:54:17,737 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 5d73bf257291417a838966121afa7c6d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-10 18:54:17,737 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 7daade243ded449c90702da866a76a59, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-10 18:54:17,737 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 2fc8ad69dbe3436abd9f185a60adc34b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-10 18:54:17,737 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting daf3147278d8400fa77df3a9eb75cd5d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-10 18:54:17,737 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 3e049f2c08d7462abfd74798d58fe2ee_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-10 18:54:17,739 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:17,740 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:17,741 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d67a27e2c80547d5b8870adbfe5978f4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d67a27e2c80547d5b8870adbfe5978f4 2023-03-10 18:54:17,743 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512310e7837f323a744f49836f2a5a2a531ea to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512310e7837f323a744f49836f2a5a2a531ea 2023-03-10 18:54:17,745 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601018b59b1b679b843c1a64861f13e618925 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601018b59b1b679b843c1a64861f13e618925 2023-03-10 18:54:17,748 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103db044b05c92c4eaf92d7c0314b4981e8 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103db044b05c92c4eaf92d7c0314b4981e8 2023-03-10 18:54:17,748 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-10 18:54:17,748 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-10 18:54:17,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:17,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:17,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:17,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:17,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:17,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:17,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:17,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:17,783 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): aaa7ef415c2e1954be313179d143a66b#family1#compaction#34 average throughput is 1.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:54:18,205 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/0ac8af29d5e942e3b44bf4f1d6c44436 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/0ac8af29d5e942e3b44bf4f1d6c44436 2023-03-10 18:54:18,211 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in aaa7ef415c2e1954be313179d143a66b/family1 of aaa7ef415c2e1954be313179d143a66b into 0ac8af29d5e942e3b44bf4f1d6c44436(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:54:18,211 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:18,212 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678474457736; duration=0sec 2023-03-10 18:54:18,212 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:18,227 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7d18e7e493d3428db94ba6ae2b8072f2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/37211e5a3b584d47ac2d3c42d211307d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3b9235826e8445889156cf618d00352d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/0c0c6c993c8d432090bc2371a9b7b473, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5e495c254a7f497398d4fd4fb79182c4, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5d73bf257291417a838966121afa7c6d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7daade243ded449c90702da866a76a59, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/2fc8ad69dbe3436abd9f185a60adc34b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/daf3147278d8400fa77df3a9eb75cd5d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3e049f2c08d7462abfd74798d58fe2ee_SeqId_44_] to archive 2023-03-10 18:54:18,228 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:18,230 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7d18e7e493d3428db94ba6ae2b8072f2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7d18e7e493d3428db94ba6ae2b8072f2 2023-03-10 18:54:18,231 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/37211e5a3b584d47ac2d3c42d211307d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/37211e5a3b584d47ac2d3c42d211307d 2023-03-10 18:54:18,233 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3b9235826e8445889156cf618d00352d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3b9235826e8445889156cf618d00352d 2023-03-10 18:54:18,234 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/0c0c6c993c8d432090bc2371a9b7b473 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/0c0c6c993c8d432090bc2371a9b7b473 2023-03-10 18:54:18,236 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5e495c254a7f497398d4fd4fb79182c4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5e495c254a7f497398d4fd4fb79182c4 2023-03-10 18:54:18,237 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5d73bf257291417a838966121afa7c6d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/5d73bf257291417a838966121afa7c6d 2023-03-10 18:54:18,238 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7daade243ded449c90702da866a76a59 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/7daade243ded449c90702da866a76a59 2023-03-10 18:54:18,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/2fc8ad69dbe3436abd9f185a60adc34b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/2fc8ad69dbe3436abd9f185a60adc34b 2023-03-10 18:54:18,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/daf3147278d8400fa77df3a9eb75cd5d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/daf3147278d8400fa77df3a9eb75cd5d 2023-03-10 18:54:18,242 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3e049f2c08d7462abfd74798d58fe2ee_SeqId_44_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3e049f2c08d7462abfd74798d58fe2ee_SeqId_44_ 2023-03-10 18:54:18,594 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512052726c229a162442b9751bf9c28d1a356 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512052726c229a162442b9751bf9c28d1a356 2023-03-10 18:54:18,595 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:18,620 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f5502318c91b45a0883478aca17a25ef first=Optional[r1] last=Optional[r2] 2023-03-10 18:54:18,620 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f5502318c91b45a0883478aca17a25ef}] 2023-03-10 18:54:18,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f5502318c91b45a0883478aca17a25ef for inclusion in aaa7ef415c2e1954be313179d143a66b/family1 2023-03-10 18:54:18,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-10 18:54:18,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:54:18,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:18,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f5502318c91b45a0883478aca17a25ef to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v94vb494iejmrta431chgshp51i4ig46ndnebanh19qtccrfgsppbpj4tvrk6niq/family1/f5502318c91b45a0883478aca17a25ef 2023-03-10 18:54:18,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v94vb494iejmrta431chgshp51i4ig46ndnebanh19qtccrfgsppbpj4tvrk6niq/family1/f5502318c91b45a0883478aca17a25ef as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3d56ad155f7e41d5a6c059b88d0299ff_SeqId_47_ 2023-03-10 18:54:18,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v94vb494iejmrta431chgshp51i4ig46ndnebanh19qtccrfgsppbpj4tvrk6niq/family1/f5502318c91b45a0883478aca17a25ef into aaa7ef415c2e1954be313179d143a66b/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3d56ad155f7e41d5a6c059b88d0299ff_SeqId_47_ - updating store file list. 2023-03-10 18:54:18,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3d56ad155f7e41d5a6c059b88d0299ff_SeqId_47_ into aaa7ef415c2e1954be313179d143a66b/family1 2023-03-10 18:54:18,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v94vb494iejmrta431chgshp51i4ig46ndnebanh19qtccrfgsppbpj4tvrk6niq/family1/f5502318c91b45a0883478aca17a25ef into aaa7ef415c2e1954be313179d143a66b/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3d56ad155f7e41d5a6c059b88d0299ff_SeqId_47_) 2023-03-10 18:54:18,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v94vb494iejmrta431chgshp51i4ig46ndnebanh19qtccrfgsppbpj4tvrk6niq/family1/f5502318c91b45a0883478aca17a25ef 2023-03-10 18:54:18,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-10 18:54:18,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-10 18:54:18,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): aaa7ef415c2e1954be313179d143a66b/family1 is initiating minor compaction (all files) 2023-03-10 18:54:18,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@225c1007; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:18,641 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of aaa7ef415c2e1954be313179d143a66b/family1 in testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:54:18,641 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/527882d24a1d414f92c4a635856c872c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/0ac8af29d5e942e3b44bf4f1d6c44436, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3d56ad155f7e41d5a6c059b88d0299ff_SeqId_47_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp, totalSize=176.8 K 2023-03-10 18:54:18,642 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 527882d24a1d414f92c4a635856c872c, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839360 2023-03-10 18:54:18,642 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 0ac8af29d5e942e3b44bf4f1d6c44436, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239360 2023-03-10 18:54:18,643 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 3d56ad155f7e41d5a6c059b88d0299ff_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965239360 2023-03-10 18:54:18,643 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:18,644 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:18,646 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201c927d871771c4438bc26b2427a7d3c78 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201c927d871771c4438bc26b2427a7d3c78 2023-03-10 18:54:18,647 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205627ddcadff5e422f9914d11c2aa5f952 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205627ddcadff5e422f9914d11c2aa5f952 2023-03-10 18:54:18,647 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-10 18:54:18,647 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-10 18:54:18,648 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:18,648 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:18,648 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:18,649 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:54:18,651 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8270 sec 2023-03-10 18:54:18,658 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): aaa7ef415c2e1954be313179d143a66b#family1#compaction#35 average throughput is 30.93 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:54:18,753 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=253 (was 263), OpenFileDescriptor=598 (was 624), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=282 (was 260) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=5545 (was 6135) 2023-03-10 18:54:18,766 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=253, OpenFileDescriptor=598, MaxFileDescriptor=60000, SystemLoadAverage=282, ProcessCount=170, AvailableMemoryMB=5544 2023-03-10 18:54:18,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-10 18:54:18,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:18,772 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:54:18,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-10 18:54:18,773 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:54:18,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-10 18:54:18,775 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4a6c534dbdc8c65fdb0cb30c4ad6a095 2023-03-10 18:54:18,775 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testScannerOnBulkLoadRefHFiles/47903cdb8627df6ff46557cb0a143884 2023-03-10 18:54:18,775 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0 2023-03-10 18:54:18,776 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4a6c534dbdc8c65fdb0cb30c4ad6a095 empty. 2023-03-10 18:54:18,776 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testScannerOnBulkLoadRefHFiles/47903cdb8627df6ff46557cb0a143884 empty. 2023-03-10 18:54:18,776 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0 empty. 2023-03-10 18:54:18,776 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4a6c534dbdc8c65fdb0cb30c4ad6a095 2023-03-10 18:54:18,776 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testScannerOnBulkLoadRefHFiles/47903cdb8627df6ff46557cb0a143884 2023-03-10 18:54:18,777 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0 2023-03-10 18:54:18,777 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-10 18:54:18,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-10 18:54:19,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-10 18:54:19,186 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-10 18:54:19,187 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4a6c534dbdc8c65fdb0cb30c4ad6a095, NAME => 'testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:54:19,187 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 47903cdb8627df6ff46557cb0a143884, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:54:19,187 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => a16c84e4f0cb02a5ce455d662efd3ad0, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:54:19,197 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:19,197 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing 4a6c534dbdc8c65fdb0cb30c4ad6a095, disabling compactions & flushes 2023-03-10 18:54:19,197 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:54:19,197 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:54:19,197 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. after waiting 0 ms 2023-03-10 18:54:19,197 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:54:19,197 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:54:19,197 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for 4a6c534dbdc8c65fdb0cb30c4ad6a095: 2023-03-10 18:54:19,198 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:19,198 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing a16c84e4f0cb02a5ce455d662efd3ad0, disabling compactions & flushes 2023-03-10 18:54:19,198 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:19,198 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:19,199 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. after waiting 0 ms 2023-03-10 18:54:19,199 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:19,199 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:19,199 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for a16c84e4f0cb02a5ce455d662efd3ad0: 2023-03-10 18:54:19,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-10 18:54:19,599 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:19,599 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 47903cdb8627df6ff46557cb0a143884, disabling compactions & flushes 2023-03-10 18:54:19,599 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:54:19,599 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:54:19,599 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. after waiting 0 ms 2023-03-10 18:54:19,599 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:54:19,599 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:54:19,599 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 47903cdb8627df6ff46557cb0a143884: 2023-03-10 18:54:19,602 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:54:19,603 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678474459602"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474459602"}]},"ts":"1678474459602"} 2023-03-10 18:54:19,603 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678474459602"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474459602"}]},"ts":"1678474459602"} 2023-03-10 18:54:19,603 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678474459602"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474459602"}]},"ts":"1678474459602"} 2023-03-10 18:54:19,606 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-10 18:54:19,606 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:54:19,607 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474459606"}]},"ts":"1678474459606"} 2023-03-10 18:54:19,608 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-10 18:54:19,612 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4a6c534dbdc8c65fdb0cb30c4ad6a095, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=47903cdb8627df6ff46557cb0a143884, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a16c84e4f0cb02a5ce455d662efd3ad0, ASSIGN}] 2023-03-10 18:54:19,614 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a16c84e4f0cb02a5ce455d662efd3ad0, ASSIGN 2023-03-10 18:54:19,614 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4a6c534dbdc8c65fdb0cb30c4ad6a095, ASSIGN 2023-03-10 18:54:19,614 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=47903cdb8627df6ff46557cb0a143884, ASSIGN 2023-03-10 18:54:19,614 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a16c84e4f0cb02a5ce455d662efd3ad0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:54:19,615 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4a6c534dbdc8c65fdb0cb30c4ad6a095, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:54:19,615 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=47903cdb8627df6ff46557cb0a143884, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:54:19,766 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=a16c84e4f0cb02a5ce455d662efd3ad0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:19,766 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=4a6c534dbdc8c65fdb0cb30c4ad6a095, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:19,766 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=47903cdb8627df6ff46557cb0a143884, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:19,766 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678474459766"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474459766"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474459766"}]},"ts":"1678474459766"} 2023-03-10 18:54:19,766 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678474459766"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474459766"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474459766"}]},"ts":"1678474459766"} 2023-03-10 18:54:19,766 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678474459766"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474459766"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474459766"}]},"ts":"1678474459766"} 2023-03-10 18:54:19,768 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=30, state=RUNNABLE; OpenRegionProcedure a16c84e4f0cb02a5ce455d662efd3ad0, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:54:19,769 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=28, state=RUNNABLE; OpenRegionProcedure 4a6c534dbdc8c65fdb0cb30c4ad6a095, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:54:19,769 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure 47903cdb8627df6ff46557cb0a143884, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:54:19,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-10 18:54:19,880 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/.tmp/family1/04c5e907dfcf44ed8c6a3b0bb73a001e as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/04c5e907dfcf44ed8c6a3b0bb73a001e 2023-03-10 18:54:19,886 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in aaa7ef415c2e1954be313179d143a66b/family1 of aaa7ef415c2e1954be313179d143a66b into 04c5e907dfcf44ed8c6a3b0bb73a001e(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:54:19,886 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:54:19,886 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1678474458641; duration=1sec 2023-03-10 18:54:19,886 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:19,919 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/527882d24a1d414f92c4a635856c872c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/0ac8af29d5e942e3b44bf4f1d6c44436, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3d56ad155f7e41d5a6c059b88d0299ff_SeqId_47_] to archive 2023-03-10 18:54:19,921 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:19,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:54:19,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 47903cdb8627df6ff46557cb0a143884, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884.', STARTKEY => '1', ENDKEY => '2'} 2023-03-10 18:54:19,923 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/527882d24a1d414f92c4a635856c872c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/527882d24a1d414f92c4a635856c872c 2023-03-10 18:54:19,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 47903cdb8627df6ff46557cb0a143884 2023-03-10 18:54:19,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:19,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 47903cdb8627df6ff46557cb0a143884 2023-03-10 18:54:19,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 47903cdb8627df6ff46557cb0a143884 2023-03-10 18:54:19,926 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/0ac8af29d5e942e3b44bf4f1d6c44436 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/0ac8af29d5e942e3b44bf4f1d6c44436 2023-03-10 18:54:19,926 INFO [StoreOpener-47903cdb8627df6ff46557cb0a143884-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 47903cdb8627df6ff46557cb0a143884 2023-03-10 18:54:19,927 DEBUG [StoreOpener-47903cdb8627df6ff46557cb0a143884-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/47903cdb8627df6ff46557cb0a143884/family1 2023-03-10 18:54:19,927 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3d56ad155f7e41d5a6c059b88d0299ff_SeqId_47_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/family1/3d56ad155f7e41d5a6c059b88d0299ff_SeqId_47_ 2023-03-10 18:54:19,928 DEBUG [StoreOpener-47903cdb8627df6ff46557cb0a143884-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/47903cdb8627df6ff46557cb0a143884/family1 2023-03-10 18:54:19,928 INFO [StoreOpener-47903cdb8627df6ff46557cb0a143884-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47903cdb8627df6ff46557cb0a143884 columnFamilyName family1 2023-03-10 18:54:19,930 INFO [StoreOpener-47903cdb8627df6ff46557cb0a143884-1] regionserver.HStore(310): Store=47903cdb8627df6ff46557cb0a143884/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:19,930 INFO [StoreOpener-47903cdb8627df6ff46557cb0a143884-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 47903cdb8627df6ff46557cb0a143884 2023-03-10 18:54:19,931 DEBUG [StoreOpener-47903cdb8627df6ff46557cb0a143884-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/47903cdb8627df6ff46557cb0a143884/family2 2023-03-10 18:54:19,932 DEBUG [StoreOpener-47903cdb8627df6ff46557cb0a143884-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/47903cdb8627df6ff46557cb0a143884/family2 2023-03-10 18:54:19,932 INFO [StoreOpener-47903cdb8627df6ff46557cb0a143884-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47903cdb8627df6ff46557cb0a143884 columnFamilyName family2 2023-03-10 18:54:19,933 INFO [StoreOpener-47903cdb8627df6ff46557cb0a143884-1] regionserver.HStore(310): Store=47903cdb8627df6ff46557cb0a143884/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:19,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/47903cdb8627df6ff46557cb0a143884 2023-03-10 18:54:19,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/47903cdb8627df6ff46557cb0a143884 2023-03-10 18:54:19,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:54:19,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 47903cdb8627df6ff46557cb0a143884 2023-03-10 18:54:19,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/47903cdb8627df6ff46557cb0a143884/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:54:19,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 47903cdb8627df6ff46557cb0a143884; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63080622, jitterRate=-0.06002548336982727}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:54:19,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 47903cdb8627df6ff46557cb0a143884: 2023-03-10 18:54:19,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884., pid=33, masterSystemTime=1678474459920 2023-03-10 18:54:19,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:54:19,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:54:19,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:54:19,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4a6c534dbdc8c65fdb0cb30c4ad6a095, NAME => 'testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095.', STARTKEY => '', ENDKEY => '1'} 2023-03-10 18:54:19,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 4a6c534dbdc8c65fdb0cb30c4ad6a095 2023-03-10 18:54:19,943 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=47903cdb8627df6ff46557cb0a143884, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:19,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:19,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4a6c534dbdc8c65fdb0cb30c4ad6a095 2023-03-10 18:54:19,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4a6c534dbdc8c65fdb0cb30c4ad6a095 2023-03-10 18:54:19,943 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678474459943"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474459943"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474459943"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474459943"}]},"ts":"1678474459943"} 2023-03-10 18:54:19,944 INFO [StoreOpener-4a6c534dbdc8c65fdb0cb30c4ad6a095-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4a6c534dbdc8c65fdb0cb30c4ad6a095 2023-03-10 18:54:19,946 DEBUG [StoreOpener-4a6c534dbdc8c65fdb0cb30c4ad6a095-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/4a6c534dbdc8c65fdb0cb30c4ad6a095/family1 2023-03-10 18:54:19,946 DEBUG [StoreOpener-4a6c534dbdc8c65fdb0cb30c4ad6a095-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/4a6c534dbdc8c65fdb0cb30c4ad6a095/family1 2023-03-10 18:54:19,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-03-10 18:54:19,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure 47903cdb8627df6ff46557cb0a143884, server=jenkins-hbase4.apache.org,43181,1678474414959 in 176 msec 2023-03-10 18:54:19,947 INFO [StoreOpener-4a6c534dbdc8c65fdb0cb30c4ad6a095-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4a6c534dbdc8c65fdb0cb30c4ad6a095 columnFamilyName family1 2023-03-10 18:54:19,948 INFO [StoreOpener-4a6c534dbdc8c65fdb0cb30c4ad6a095-1] regionserver.HStore(310): Store=4a6c534dbdc8c65fdb0cb30c4ad6a095/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:19,948 INFO [StoreOpener-4a6c534dbdc8c65fdb0cb30c4ad6a095-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4a6c534dbdc8c65fdb0cb30c4ad6a095 2023-03-10 18:54:19,948 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=47903cdb8627df6ff46557cb0a143884, ASSIGN in 335 msec 2023-03-10 18:54:19,950 DEBUG [StoreOpener-4a6c534dbdc8c65fdb0cb30c4ad6a095-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/4a6c534dbdc8c65fdb0cb30c4ad6a095/family2 2023-03-10 18:54:19,950 DEBUG [StoreOpener-4a6c534dbdc8c65fdb0cb30c4ad6a095-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/4a6c534dbdc8c65fdb0cb30c4ad6a095/family2 2023-03-10 18:54:19,950 INFO [StoreOpener-4a6c534dbdc8c65fdb0cb30c4ad6a095-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4a6c534dbdc8c65fdb0cb30c4ad6a095 columnFamilyName family2 2023-03-10 18:54:19,951 INFO [StoreOpener-4a6c534dbdc8c65fdb0cb30c4ad6a095-1] regionserver.HStore(310): Store=4a6c534dbdc8c65fdb0cb30c4ad6a095/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:19,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/4a6c534dbdc8c65fdb0cb30c4ad6a095 2023-03-10 18:54:19,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/4a6c534dbdc8c65fdb0cb30c4ad6a095 2023-03-10 18:54:19,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:54:19,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4a6c534dbdc8c65fdb0cb30c4ad6a095 2023-03-10 18:54:19,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/4a6c534dbdc8c65fdb0cb30c4ad6a095/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:54:19,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4a6c534dbdc8c65fdb0cb30c4ad6a095; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64103560, jitterRate=-0.04478251934051514}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:54:19,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4a6c534dbdc8c65fdb0cb30c4ad6a095: 2023-03-10 18:54:19,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095., pid=32, masterSystemTime=1678474459920 2023-03-10 18:54:19,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:54:19,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:54:19,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:19,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a16c84e4f0cb02a5ce455d662efd3ad0, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0.', STARTKEY => '2', ENDKEY => ''} 2023-03-10 18:54:19,961 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=4a6c534dbdc8c65fdb0cb30c4ad6a095, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:19,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles a16c84e4f0cb02a5ce455d662efd3ad0 2023-03-10 18:54:19,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:19,961 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678474459961"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474459961"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474459961"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474459961"}]},"ts":"1678474459961"} 2023-03-10 18:54:19,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a16c84e4f0cb02a5ce455d662efd3ad0 2023-03-10 18:54:19,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a16c84e4f0cb02a5ce455d662efd3ad0 2023-03-10 18:54:19,962 INFO [StoreOpener-a16c84e4f0cb02a5ce455d662efd3ad0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a16c84e4f0cb02a5ce455d662efd3ad0 2023-03-10 18:54:19,964 DEBUG [StoreOpener-a16c84e4f0cb02a5ce455d662efd3ad0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1 2023-03-10 18:54:19,964 DEBUG [StoreOpener-a16c84e4f0cb02a5ce455d662efd3ad0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1 2023-03-10 18:54:19,965 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-03-10 18:54:19,965 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; OpenRegionProcedure 4a6c534dbdc8c65fdb0cb30c4ad6a095, server=jenkins-hbase4.apache.org,43181,1678474414959 in 194 msec 2023-03-10 18:54:19,965 INFO [StoreOpener-a16c84e4f0cb02a5ce455d662efd3ad0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a16c84e4f0cb02a5ce455d662efd3ad0 columnFamilyName family1 2023-03-10 18:54:19,966 INFO [StoreOpener-a16c84e4f0cb02a5ce455d662efd3ad0-1] regionserver.HStore(310): Store=a16c84e4f0cb02a5ce455d662efd3ad0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:19,966 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4a6c534dbdc8c65fdb0cb30c4ad6a095, ASSIGN in 353 msec 2023-03-10 18:54:19,966 INFO [StoreOpener-a16c84e4f0cb02a5ce455d662efd3ad0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a16c84e4f0cb02a5ce455d662efd3ad0 2023-03-10 18:54:19,968 DEBUG [StoreOpener-a16c84e4f0cb02a5ce455d662efd3ad0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family2 2023-03-10 18:54:19,968 DEBUG [StoreOpener-a16c84e4f0cb02a5ce455d662efd3ad0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family2 2023-03-10 18:54:19,968 INFO [StoreOpener-a16c84e4f0cb02a5ce455d662efd3ad0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a16c84e4f0cb02a5ce455d662efd3ad0 columnFamilyName family2 2023-03-10 18:54:19,969 INFO [StoreOpener-a16c84e4f0cb02a5ce455d662efd3ad0-1] regionserver.HStore(310): Store=a16c84e4f0cb02a5ce455d662efd3ad0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:19,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0 2023-03-10 18:54:19,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0 2023-03-10 18:54:19,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:54:19,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a16c84e4f0cb02a5ce455d662efd3ad0 2023-03-10 18:54:19,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:54:19,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a16c84e4f0cb02a5ce455d662efd3ad0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69485569, jitterRate=0.035415664315223694}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:54:19,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a16c84e4f0cb02a5ce455d662efd3ad0: 2023-03-10 18:54:19,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0., pid=31, masterSystemTime=1678474459920 2023-03-10 18:54:19,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:19,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:19,980 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=a16c84e4f0cb02a5ce455d662efd3ad0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:19,981 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678474459980"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474459980"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474459980"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474459980"}]},"ts":"1678474459980"} 2023-03-10 18:54:19,984 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=30 2023-03-10 18:54:19,984 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=30, state=SUCCESS; OpenRegionProcedure a16c84e4f0cb02a5ce455d662efd3ad0, server=jenkins-hbase4.apache.org,43181,1678474414959 in 214 msec 2023-03-10 18:54:19,986 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-03-10 18:54:19,986 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a16c84e4f0cb02a5ce455d662efd3ad0, ASSIGN in 372 msec 2023-03-10 18:54:19,987 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:54:19,987 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474459987"}]},"ts":"1678474459987"} 2023-03-10 18:54:19,988 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-10 18:54:19,991 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:54:19,992 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2220 sec 2023-03-10 18:54:20,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-10 18:54:20,880 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-10 18:54:20,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:20,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:20,889 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-10 18:54:20,889 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:20,889 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-10 18:54:20,889 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-10 18:54:20,890 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:20,890 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:20,891 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:20,891 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:20,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:20,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:20,892 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:20,892 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:20,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:20,892 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:20,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:20,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:20,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:20,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:20,893 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-10 18:54:20,893 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:20,893 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-10 18:54:20,893 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:20,893 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. started... 2023-03-10 18:54:20,893 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:54:20,893 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:54:20,894 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:54:20,893 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. started... 2023-03-10 18:54:20,894 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. started... 2023-03-10 18:54:20,894 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a16c84e4f0cb02a5ce455d662efd3ad0 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-10 18:54:20,894 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:20,895 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4a6c534dbdc8c65fdb0cb30c4ad6a095: 2023-03-10 18:54:20,895 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 47903cdb8627df6ff46557cb0a143884: 2023-03-10 18:54:20,895 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:54:20,895 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:54:20,895 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:54:20,895 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:54:21,311 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310618cdcef32654c0fb061ee582ea91df8 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230310618cdcef32654c0fb061ee582ea91df8 2023-03-10 18:54:21,717 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/.tmp/family1/fa6b94ff220b4956802770448ded0b07 2023-03-10 18:54:21,722 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/.tmp/family1/fa6b94ff220b4956802770448ded0b07 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/fa6b94ff220b4956802770448ded0b07 2023-03-10 18:54:21,727 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/fa6b94ff220b4956802770448ded0b07, entries=1, sequenceid=5, filesize=4.9 K 2023-03-10 18:54:21,728 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for a16c84e4f0cb02a5ce455d662efd3ad0 in 834ms, sequenceid=5, compaction requested=false 2023-03-10 18:54:21,728 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-10 18:54:21,729 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a16c84e4f0cb02a5ce455d662efd3ad0: 2023-03-10 18:54:21,729 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:21,729 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:54:21,729 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:54:21,729 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,729 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-10 18:54:21,729 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-10 18:54:21,731 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,731 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:21,731 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,731 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:21,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:21,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:21,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-10 18:54:21,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70826bfc[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:21,733 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-10 18:54:21,733 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,735 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,735 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-10 18:54:21,735 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-10 18:54:21,735 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,735 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:21,735 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:21,737 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-10 18:54:21,737 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,737 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:21,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:21,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:21,737 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-10 18:54:21,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:21,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:21,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:21,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:21,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-10 18:54:21,739 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-10 18:54:21,739 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:21,739 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:21,739 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,739 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:21,741 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,741 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:21,741 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:21,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:21,742 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:21,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:21,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,743 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:21,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,750 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,751 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:21,751 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:21,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:21,751 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:21,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-10 18:54:21,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:21,751 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,751 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-10 18:54:21,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:21,751 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,751 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,751 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,751 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,751 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,751 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:21,751 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:21,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:21,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:21,852 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:21,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:21,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:21,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:21,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,858 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-10 18:54:21,858 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:21,858 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-10 18:54:21,858 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-10 18:54:21,858 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,858 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,860 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:21,860 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,860 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:21,860 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:21,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:21,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,861 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-10 18:54:21,861 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:21,861 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-10 18:54:21,861 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:21,861 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-10 18:54:21,862 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:21,862 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. started... 2023-03-10 18:54:21,862 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a16c84e4f0cb02a5ce455d662efd3ad0 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-10 18:54:21,862 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:54:21,862 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. started... 2023-03-10 18:54:21,862 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 47903cdb8627df6ff46557cb0a143884: 2023-03-10 18:54:21,862 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:54:21,862 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:54:21,862 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:21,862 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:54:21,862 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. started... 2023-03-10 18:54:21,862 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:54:21,863 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4a6c534dbdc8c65fdb0cb30c4ad6a095: 2023-03-10 18:54:21,863 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:54:21,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:54:22,275 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310b1c3c527ce1a40c3bba8b29a339bd7cd to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230310b1c3c527ce1a40c3bba8b29a339bd7cd 2023-03-10 18:54:22,681 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/.tmp/family1/05e3058b15eb4a038d9f0a47c60b073a 2023-03-10 18:54:22,687 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/.tmp/family1/05e3058b15eb4a038d9f0a47c60b073a as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/05e3058b15eb4a038d9f0a47c60b073a 2023-03-10 18:54:22,691 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/05e3058b15eb4a038d9f0a47c60b073a, entries=2, sequenceid=9, filesize=5.0 K 2023-03-10 18:54:22,692 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for a16c84e4f0cb02a5ce455d662efd3ad0 in 830ms, sequenceid=9, compaction requested=false 2023-03-10 18:54:22,693 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a16c84e4f0cb02a5ce455d662efd3ad0: 2023-03-10 18:54:22,693 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:22,693 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:54:22,693 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:54:22,693 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,693 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-10 18:54:22,693 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-10 18:54:22,695 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,695 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:22,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:22,695 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,695 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:22,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:22,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:22,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:22,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-10 18:54:22,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c01db6a[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:22,697 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-10 18:54:22,697 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,700 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-10 18:54:22,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-10 18:54:22,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:22,700 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,700 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:22,702 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,702 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-10 18:54:22,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:22,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:22,702 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:22,702 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-10 18:54:22,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:22,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:22,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:22,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:22,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-10 18:54:22,704 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-10 18:54:22,704 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:22,704 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:22,704 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,704 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:22,705 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,705 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,705 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:22,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:22,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:22,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:22,706 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:22,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:22,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:22,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,707 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:22,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,709 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,709 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:22,709 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,709 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:22,709 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,709 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:22,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:22,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:22,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-10 18:54:22,710 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:22,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-10 18:54:22,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:22,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:22,709 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:22,710 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,710 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:22,710 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:22,710 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,710 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,810 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:22,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:22,817 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-10 18:54:22,819 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-10 18:54:22,820 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-10 18:54:22,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:22,821 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475062821 2023-03-10 18:54:22,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:54:22,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-10 18:54:22,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-10 18:54:22,822 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x63825d30 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:54:22,828 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f2f1f45, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:54:22,828 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230310 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-10 18:54:23,655 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310fae85e6bbc3a47ae91f9b32878049271 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230310fae85e6bbc3a47ae91f9b32878049271 2023-03-10 18:54:23,656 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230310 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-10 18:54:23,671 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:54:23,672 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41554, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:54:23,679 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:54:23,680 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43164, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:54:23,689 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230310/family1/31ef0ac50f8145cf8112f3382140c61e first=Optional[k0] last=Optional[k1] 2023-03-10 18:54:23,690 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230310/family1/31ef0ac50f8145cf8112f3382140c61e}] 2023-03-10 18:54:23,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230310/family1/31ef0ac50f8145cf8112f3382140c61e for inclusion in a16c84e4f0cb02a5ce455d662efd3ad0/family1 2023-03-10 18:54:23,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-10 18:54:23,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-10 18:54:23,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for a16c84e4f0cb02a5ce455d662efd3ad0: 2023-03-10 18:54:23,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230310/family1/31ef0ac50f8145cf8112f3382140c61e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testScannerOnBulkLoadRefHFiles__kbnkv99hc1htplhobiolh7fo2tnj3audo00mk9jpoaqfl2lio71qu1cgfv7jqq94/family1/31ef0ac50f8145cf8112f3382140c61e 2023-03-10 18:54:23,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testScannerOnBulkLoadRefHFiles__kbnkv99hc1htplhobiolh7fo2tnj3audo00mk9jpoaqfl2lio71qu1cgfv7jqq94/family1/31ef0ac50f8145cf8112f3382140c61e as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/081b4804c4eb4175a0fdd08f470a1d2a_SeqId_12_ 2023-03-10 18:54:23,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testScannerOnBulkLoadRefHFiles__kbnkv99hc1htplhobiolh7fo2tnj3audo00mk9jpoaqfl2lio71qu1cgfv7jqq94/family1/31ef0ac50f8145cf8112f3382140c61e into a16c84e4f0cb02a5ce455d662efd3ad0/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/081b4804c4eb4175a0fdd08f470a1d2a_SeqId_12_ - updating store file list. 2023-03-10 18:54:23,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/081b4804c4eb4175a0fdd08f470a1d2a_SeqId_12_ into a16c84e4f0cb02a5ce455d662efd3ad0/family1 2023-03-10 18:54:23,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testScannerOnBulkLoadRefHFiles__kbnkv99hc1htplhobiolh7fo2tnj3audo00mk9jpoaqfl2lio71qu1cgfv7jqq94/family1/31ef0ac50f8145cf8112f3382140c61e into a16c84e4f0cb02a5ce455d662efd3ad0/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/081b4804c4eb4175a0fdd08f470a1d2a_SeqId_12_) 2023-03-10 18:54:23,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testScannerOnBulkLoadRefHFiles__kbnkv99hc1htplhobiolh7fo2tnj3audo00mk9jpoaqfl2lio71qu1cgfv7jqq94/family1/31ef0ac50f8145cf8112f3382140c61e 2023-03-10 18:54:23,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-10 18:54:23,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-10 18:54:23,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): a16c84e4f0cb02a5ce455d662efd3ad0/family1 is initiating minor compaction (all files) 2023-03-10 18:54:23,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@21c02400; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:23,708 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a16c84e4f0cb02a5ce455d662efd3ad0/family1 in testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:54:23,708 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/fa6b94ff220b4956802770448ded0b07, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/05e3058b15eb4a038d9f0a47c60b073a, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/081b4804c4eb4175a0fdd08f470a1d2a_SeqId_12_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/.tmp, totalSize=15.1 K 2023-03-10 18:54:23,709 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting fa6b94ff220b4956802770448ded0b07, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474460880 2023-03-10 18:54:23,709 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 05e3058b15eb4a038d9f0a47c60b073a, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474460880 2023-03-10 18:54:23,710 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 081b4804c4eb4175a0fdd08f470a1d2a_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474460880 2023-03-10 18:54:23,710 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230310 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-10 18:54:23,711 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:23,712 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230310618cdcef32654c0fb061ee582ea91df8 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230310618cdcef32654c0fb061ee582ea91df8 2023-03-10 18:54:23,713 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230310b1c3c527ce1a40c3bba8b29a339bd7cd to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230310b1c3c527ce1a40c3bba8b29a339bd7cd 2023-03-10 18:54:23,714 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-10 18:54:23,714 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230310' 2023-03-10 18:54:23,714 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230310 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-10 18:54:23,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-10 18:54:23,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:23,715 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-10 18:54:23,717 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 896 msec 2023-03-10 18:54:23,719 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): a16c84e4f0cb02a5ce455d662efd3ad0#family1#compaction#38 average throughput is 0.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:54:23,746 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=251 (was 253), OpenFileDescriptor=591 (was 598), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=275 (was 282), ProcessCount=170 (was 170), AvailableMemoryMB=5618 (was 5544) - AvailableMemoryMB LEAK? - 2023-03-10 18:54:23,757 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=251, OpenFileDescriptor=591, MaxFileDescriptor=60000, SystemLoadAverage=275, ProcessCount=170, AvailableMemoryMB=5618 2023-03-10 18:54:23,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-10 18:54:23,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:23,761 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:54:23,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-10 18:54:23,762 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:54:23,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-10 18:54:23,763 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830 2023-03-10 18:54:23,764 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830 empty. 2023-03-10 18:54:23,764 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830 2023-03-10 18:54:23,764 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-10 18:54:23,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-10 18:54:24,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-10 18:54:24,172 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-10 18:54:24,173 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => b416ae27480b11a339c44b606dbd5830, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:54:24,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-10 18:54:24,579 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:24,579 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing b416ae27480b11a339c44b606dbd5830, disabling compactions & flushes 2023-03-10 18:54:24,579 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:24,579 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:24,579 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. after waiting 0 ms 2023-03-10 18:54:24,579 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:24,579 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:24,579 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:24,582 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:54:24,583 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678474464583"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474464583"}]},"ts":"1678474464583"} 2023-03-10 18:54:24,584 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-10 18:54:24,585 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:54:24,585 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474464585"}]},"ts":"1678474464585"} 2023-03-10 18:54:24,586 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-10 18:54:24,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=b416ae27480b11a339c44b606dbd5830, ASSIGN}] 2023-03-10 18:54:24,590 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=b416ae27480b11a339c44b606dbd5830, ASSIGN 2023-03-10 18:54:24,591 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=b416ae27480b11a339c44b606dbd5830, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:54:24,742 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=b416ae27480b11a339c44b606dbd5830, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:24,742 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678474464742"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474464742"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474464742"}]},"ts":"1678474464742"} 2023-03-10 18:54:24,744 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure b416ae27480b11a339c44b606dbd5830, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:54:24,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-10 18:54:24,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:24,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b416ae27480b11a339c44b606dbd5830, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:54:24,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy b416ae27480b11a339c44b606dbd5830 2023-03-10 18:54:24,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:24,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b416ae27480b11a339c44b606dbd5830 2023-03-10 18:54:24,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b416ae27480b11a339c44b606dbd5830 2023-03-10 18:54:24,900 INFO [StoreOpener-b416ae27480b11a339c44b606dbd5830-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b416ae27480b11a339c44b606dbd5830 2023-03-10 18:54:24,901 DEBUG [StoreOpener-b416ae27480b11a339c44b606dbd5830-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1 2023-03-10 18:54:24,901 DEBUG [StoreOpener-b416ae27480b11a339c44b606dbd5830-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1 2023-03-10 18:54:24,902 INFO [StoreOpener-b416ae27480b11a339c44b606dbd5830-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b416ae27480b11a339c44b606dbd5830 columnFamilyName family1 2023-03-10 18:54:24,903 INFO [StoreOpener-b416ae27480b11a339c44b606dbd5830-1] regionserver.HStore(310): Store=b416ae27480b11a339c44b606dbd5830/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:24,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830 2023-03-10 18:54:24,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830 2023-03-10 18:54:24,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b416ae27480b11a339c44b606dbd5830 2023-03-10 18:54:24,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:54:24,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b416ae27480b11a339c44b606dbd5830; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68250047, jitterRate=0.01700495183467865}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:54:24,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:24,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830., pid=37, masterSystemTime=1678474464895 2023-03-10 18:54:24,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:24,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:24,911 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=b416ae27480b11a339c44b606dbd5830, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:24,911 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678474464911"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474464911"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474464911"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474464911"}]},"ts":"1678474464911"} 2023-03-10 18:54:24,914 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-10 18:54:24,914 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure b416ae27480b11a339c44b606dbd5830, server=jenkins-hbase4.apache.org,43181,1678474414959 in 168 msec 2023-03-10 18:54:24,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-10 18:54:24,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=b416ae27480b11a339c44b606dbd5830, ASSIGN in 325 msec 2023-03-10 18:54:24,916 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:54:24,916 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474464916"}]},"ts":"1678474464916"} 2023-03-10 18:54:24,917 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-10 18:54:24,920 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:54:24,921 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1610 sec 2023-03-10 18:54:24,940 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/.tmp/family1/7dafcbc7c21d462492625eb6139ca85e as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/7dafcbc7c21d462492625eb6139ca85e 2023-03-10 18:54:24,945 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a16c84e4f0cb02a5ce455d662efd3ad0/family1 of a16c84e4f0cb02a5ce455d662efd3ad0 into 7dafcbc7c21d462492625eb6139ca85e(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:54:24,946 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a16c84e4f0cb02a5ce455d662efd3ad0: 2023-03-10 18:54:24,946 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1678474463708; duration=1sec 2023-03-10 18:54:24,946 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:25,015 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/fa6b94ff220b4956802770448ded0b07, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/05e3058b15eb4a038d9f0a47c60b073a, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/081b4804c4eb4175a0fdd08f470a1d2a_SeqId_12_] to archive 2023-03-10 18:54:25,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:25,018 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/fa6b94ff220b4956802770448ded0b07 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/fa6b94ff220b4956802770448ded0b07 2023-03-10 18:54:25,019 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/05e3058b15eb4a038d9f0a47c60b073a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/05e3058b15eb4a038d9f0a47c60b073a 2023-03-10 18:54:25,020 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/081b4804c4eb4175a0fdd08f470a1d2a_SeqId_12_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/family1/081b4804c4eb4175a0fdd08f470a1d2a_SeqId_12_ 2023-03-10 18:54:25,453 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:54:25,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-10 18:54:25,868 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-10 18:54:25,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:25,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:25,892 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:25,892 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:25,892 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:25,892 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:25,893 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:25,893 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:25,895 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:25,895 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:25,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:25,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:25,895 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:25,895 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:25,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:25,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:25,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:25,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:25,896 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:25,896 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:25,896 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:25,896 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:25,896 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:25,897 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:25,897 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. started... 2023-03-10 18:54:25,897 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b416ae27480b11a339c44b606dbd5830 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-10 18:54:26,320 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511302aaeb17143c34b09be35addc5af1d567 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511302aaeb17143c34b09be35addc5af1d567 2023-03-10 18:54:26,723 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/66bcbd9823c643bda191f24359cf5bac 2023-03-10 18:54:26,729 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/66bcbd9823c643bda191f24359cf5bac as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/66bcbd9823c643bda191f24359cf5bac 2023-03-10 18:54:26,733 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/66bcbd9823c643bda191f24359cf5bac, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-10 18:54:26,734 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for b416ae27480b11a339c44b606dbd5830 in 836ms, sequenceid=5, compaction requested=false 2023-03-10 18:54:26,734 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:26,734 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:26,734 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:26,734 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:26,734 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:26,734 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,734 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:26,734 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:26,736 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,736 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:26,736 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,736 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:26,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:26,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:26,737 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:26,737 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3fb5f878[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:26,737 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:26,737 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,739 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,739 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:26,739 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,739 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:26,739 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:26,739 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:26,740 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:26,740 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:26,740 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,740 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:26,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:26,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:26,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:26,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:26,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:26,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:26,743 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:26,743 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:26,743 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:26,743 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:26,743 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,743 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:26,748 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,748 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:26,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:26,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,748 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:26,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:26,748 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:26,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:26,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:26,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,749 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:26,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,752 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,752 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,752 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:26,752 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:26,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:26,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:26,752 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-10 18:54:26,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:26,752 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:26,752 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,752 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,752 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,753 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,752 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:26,753 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:26,753 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:26,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:26,853 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:26,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:26,859 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:26,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:26,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,861 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:26,861 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:26,861 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:26,861 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:26,861 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,861 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,863 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,863 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:26,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:26,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:26,863 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:26,863 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:26,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:26,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,864 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:26,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:26,864 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:26,864 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:26,864 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:26,865 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:26,865 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:26,865 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:26,865 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. started... 2023-03-10 18:54:26,865 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b416ae27480b11a339c44b606dbd5830 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-10 18:54:27,278 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112863bf47dd491d44f88262619420185082 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112863bf47dd491d44f88262619420185082 2023-03-10 18:54:27,684 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/bc9ea7bb586e4688aeffe0fbe42dbe0e 2023-03-10 18:54:27,689 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/bc9ea7bb586e4688aeffe0fbe42dbe0e as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/bc9ea7bb586e4688aeffe0fbe42dbe0e 2023-03-10 18:54:27,692 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/bc9ea7bb586e4688aeffe0fbe42dbe0e, entries=1, sequenceid=9, filesize=5.0 K 2023-03-10 18:54:27,693 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for b416ae27480b11a339c44b606dbd5830 in 828ms, sequenceid=9, compaction requested=false 2023-03-10 18:54:27,694 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:27,694 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:27,694 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:27,694 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:27,694 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,694 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:27,694 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:27,696 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,696 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:27,696 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,696 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39449913[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:27,698 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:27,698 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,702 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,702 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:27,702 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:27,703 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:27,703 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,703 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:27,704 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:27,704 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,704 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:27,704 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:27,706 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:27,706 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:27,706 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:27,706 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,706 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:27,708 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,708 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:27,708 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:27,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:27,709 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,709 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,712 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,712 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:27,712 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:27,712 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:27,712 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:27,712 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,712 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:27,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:27,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:27,712 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,712 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,712 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,713 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-10 18:54:27,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:27,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:27,713 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:27,713 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:27,813 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:27,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:27,818 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:27,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:27,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,820 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:27,820 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:27,820 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:27,820 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:27,820 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,820 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,822 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,822 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:27,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:27,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:27,822 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:27,822 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:27,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:27,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,823 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,823 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:27,823 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:27,823 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:27,823 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:27,823 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:27,824 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:27,824 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:27,824 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:27,824 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. started... 2023-03-10 18:54:27,824 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b416ae27480b11a339c44b606dbd5830 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:28,236 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201337e3e71c59a41afaefe88c3b5065037 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201337e3e71c59a41afaefe88c3b5065037 2023-03-10 18:54:28,642 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/fc0bab1f2a924cd1b934b35c12d03b3c 2023-03-10 18:54:28,647 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/fc0bab1f2a924cd1b934b35c12d03b3c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/fc0bab1f2a924cd1b934b35c12d03b3c 2023-03-10 18:54:28,651 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/fc0bab1f2a924cd1b934b35c12d03b3c, entries=1, sequenceid=13, filesize=4.9 K 2023-03-10 18:54:28,652 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b416ae27480b11a339c44b606dbd5830 in 828ms, sequenceid=13, compaction requested=true 2023-03-10 18:54:28,652 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:28,652 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:28,654 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,654 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:28,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:28,654 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,654 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:28,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:28,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:28,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:28,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:28,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5afac84c[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:28,656 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:28,656 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,657 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,657 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:28,657 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:28,657 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:28,657 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,657 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:28,660 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,660 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,660 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:28,660 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:28,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:28,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:28,662 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:28,662 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:28,662 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:28,662 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,662 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:28,664 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,664 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:28,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:28,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,664 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:28,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:28,665 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:28,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:28,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:28,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,666 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:28,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,668 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,668 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,668 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:28,668 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:28,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:28,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:28,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:28,668 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:28,668 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:28,668 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,668 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,668 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:28,669 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:28,668 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:28,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:28,669 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,769 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:28,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:28,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:28,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:28,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,775 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:28,775 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:28,775 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:28,775 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:28,776 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,776 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,777 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:28,777 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:28,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:28,778 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:28,778 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:28,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:28,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,778 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:28,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:28,778 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:28,779 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:28,779 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:28,779 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:28,779 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:28,779 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:28,779 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. started... 2023-03-10 18:54:28,779 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b416ae27480b11a339c44b606dbd5830 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:29,192 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205fd2f313567a04fa0bc7e44be9a776de0 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205fd2f313567a04fa0bc7e44be9a776de0 2023-03-10 18:54:29,598 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/3d63da2188ae4482a9ef1b9339263d1b 2023-03-10 18:54:29,603 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/3d63da2188ae4482a9ef1b9339263d1b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/3d63da2188ae4482a9ef1b9339263d1b 2023-03-10 18:54:29,607 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/3d63da2188ae4482a9ef1b9339263d1b, entries=1, sequenceid=17, filesize=4.9 K 2023-03-10 18:54:29,608 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b416ae27480b11a339c44b606dbd5830 in 829ms, sequenceid=17, compaction requested=true 2023-03-10 18:54:29,609 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:29,609 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:29,609 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:29,609 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:29,609 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,609 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:29,609 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:29,611 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,611 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:29,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:29,611 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,611 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:29,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:29,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:29,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:29,612 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:29,612 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@780292f2[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:29,612 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:29,612 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,613 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,614 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:29,614 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:29,614 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:29,614 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,614 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:29,615 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,615 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:29,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:29,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:29,615 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:29,615 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:29,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:29,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:29,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:29,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:29,617 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:29,617 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:29,617 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:29,617 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:29,617 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,617 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:29,620 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,620 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:29,621 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:29,621 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,622 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,624 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,624 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:29,624 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:29,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:29,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:29,624 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:29,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:29,624 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,624 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:29,624 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,625 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,625 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,625 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,625 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:29,625 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:29,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:29,725 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:29,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:29,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:29,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:29,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,732 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:29,732 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:29,732 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:29,732 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:29,733 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,733 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,735 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,735 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:29,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:29,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:29,735 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:29,735 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:29,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:29,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:29,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:29,736 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:29,736 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:29,736 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:29,737 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:29,737 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:29,737 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:29,737 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. started... 2023-03-10 18:54:29,737 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b416ae27480b11a339c44b606dbd5830 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:30,152 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122838d15f31f34b44e1a445ceef72f998a7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122838d15f31f34b44e1a445ceef72f998a7 2023-03-10 18:54:30,401 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:54:30,558 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/6ded93454b4c4991ac39da3243112db2 2023-03-10 18:54:30,563 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/6ded93454b4c4991ac39da3243112db2 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/6ded93454b4c4991ac39da3243112db2 2023-03-10 18:54:30,567 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/6ded93454b4c4991ac39da3243112db2, entries=1, sequenceid=21, filesize=4.9 K 2023-03-10 18:54:30,568 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b416ae27480b11a339c44b606dbd5830 in 831ms, sequenceid=21, compaction requested=true 2023-03-10 18:54:30,568 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:30,568 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:30,568 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:30,568 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:30,568 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,568 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:30,568 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:30,574 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,574 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:30,574 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,574 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:30,575 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:30,575 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68e4d3b2[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:30,575 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:30,575 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,577 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,577 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:30,577 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:30,577 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,577 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:30,577 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:30,578 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,578 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:30,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:30,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:30,578 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:30,579 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:30,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:30,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:30,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:30,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:30,580 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:30,580 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:30,581 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:30,581 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:30,581 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,581 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:30,583 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,583 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:30,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:30,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,583 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:30,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:30,584 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:30,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:30,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:30,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,585 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:30,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,587 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,587 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:30,587 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:30,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-10 18:54:30,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:30,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:30,587 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:30,587 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:30,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:30,587 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,588 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,588 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,588 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,588 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:30,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:30,588 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:30,588 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:30,689 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:30,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:30,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:30,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:30,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,695 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:30,696 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:30,696 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:30,696 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:30,696 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,696 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,697 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:30,697 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:30,698 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:30,698 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:30,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,698 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:30,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,698 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:30,699 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:30,699 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:30,699 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:30,699 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:30,699 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:30,699 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:30,699 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:30,699 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. started... 2023-03-10 18:54:30,699 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b416ae27480b11a339c44b606dbd5830 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:31,113 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123184af2887fc5845c6a6318c9196115937 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123184af2887fc5845c6a6318c9196115937 2023-03-10 18:54:31,519 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/25928ec7c38a42dc8b9911e2f196d39b 2023-03-10 18:54:31,524 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/25928ec7c38a42dc8b9911e2f196d39b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/25928ec7c38a42dc8b9911e2f196d39b 2023-03-10 18:54:31,527 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/25928ec7c38a42dc8b9911e2f196d39b, entries=1, sequenceid=25, filesize=4.9 K 2023-03-10 18:54:31,528 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b416ae27480b11a339c44b606dbd5830 in 829ms, sequenceid=25, compaction requested=true 2023-03-10 18:54:31,528 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:31,529 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:31,529 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:31,529 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:31,529 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,529 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:31,529 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:31,530 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,530 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:31,531 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,531 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@757543bc[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:31,532 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:31,532 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,534 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,535 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:31,535 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:31,535 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:31,535 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,535 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:31,536 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:31,536 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:31,536 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,536 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:31,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:31,538 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:31,538 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:31,538 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:31,538 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,538 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:31,540 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,540 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:31,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:31,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,540 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:31,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:31,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:31,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:31,540 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:31,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,541 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:31,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,544 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,544 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:31,544 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:31,544 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,544 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,544 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:31,544 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:31,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-10 18:54:31,544 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,544 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:31,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:31,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:31,544 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,545 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:31,545 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:31,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:31,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:31,645 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:31,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:31,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:31,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:31,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,652 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:31,652 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:31,652 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:31,652 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:31,652 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,652 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,654 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,654 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:31,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:31,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:31,655 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:31,655 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:31,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:31,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,655 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:31,656 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:31,656 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:31,656 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:31,656 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:31,656 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:31,656 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:31,656 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:31,656 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. started... 2023-03-10 18:54:31,656 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b416ae27480b11a339c44b606dbd5830 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:32,069 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101eedfea0407a649f7bfbeb10d2b948b15 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101eedfea0407a649f7bfbeb10d2b948b15 2023-03-10 18:54:32,478 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/2eb0b443b2d14b86a266e9369c54e508 2023-03-10 18:54:32,483 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/2eb0b443b2d14b86a266e9369c54e508 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/2eb0b443b2d14b86a266e9369c54e508 2023-03-10 18:54:32,487 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/2eb0b443b2d14b86a266e9369c54e508, entries=1, sequenceid=29, filesize=4.9 K 2023-03-10 18:54:32,488 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b416ae27480b11a339c44b606dbd5830 in 832ms, sequenceid=29, compaction requested=true 2023-03-10 18:54:32,488 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:32,488 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:32,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:32,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:32,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:32,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:32,490 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,490 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:32,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:32,490 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,490 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:32,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:32,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:32,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:32,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:32,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79414df6[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:32,492 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:32,492 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,493 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,493 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:32,493 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,493 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:32,493 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:32,493 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:32,495 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,495 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:32,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:32,495 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:32,496 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:32,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:32,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:32,498 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:32,498 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:32,498 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:32,498 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,498 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:32,500 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,500 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:32,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:32,500 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:32,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:32,501 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:32,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:32,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:32,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,502 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:32,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:32,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:32,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:32,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:32,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:32,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:32,504 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:32,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:32,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:32,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,504 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:32,504 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-10 18:54:32,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,504 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:32,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,505 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,505 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,605 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:32,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:32,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:32,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:32,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,611 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:32,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:32,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:32,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:32,612 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,612 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,616 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,616 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:32,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:32,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:32,616 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:32,616 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:32,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:32,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,617 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,617 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:32,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:32,617 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:32,617 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:32,617 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:32,617 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:32,617 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:32,617 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:32,617 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. started... 2023-03-10 18:54:32,617 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b416ae27480b11a339c44b606dbd5830 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:33,030 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103fcffa4d696354ea7a77e3920f7a94a56 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103fcffa4d696354ea7a77e3920f7a94a56 2023-03-10 18:54:33,435 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/e205d865714145b6a1d02ea25ff0b30e 2023-03-10 18:54:33,440 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/e205d865714145b6a1d02ea25ff0b30e as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/e205d865714145b6a1d02ea25ff0b30e 2023-03-10 18:54:33,444 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/e205d865714145b6a1d02ea25ff0b30e, entries=1, sequenceid=33, filesize=4.9 K 2023-03-10 18:54:33,445 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b416ae27480b11a339c44b606dbd5830 in 828ms, sequenceid=33, compaction requested=true 2023-03-10 18:54:33,445 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:33,446 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:33,446 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:33,446 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:33,446 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,446 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:33,446 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:33,447 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,447 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:33,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:33,448 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,448 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:33,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:33,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:33,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:33,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:33,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15b8d52e[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:33,449 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:33,449 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,450 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,450 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:33,451 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:33,451 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,451 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:33,451 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:33,453 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:33,453 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,453 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:33,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:33,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:33,453 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:33,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:33,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:33,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:33,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:33,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:33,455 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:33,455 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:33,455 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:33,455 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,455 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:33,457 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,457 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:33,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,457 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:33,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:33,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:33,457 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:33,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:33,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:33,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,458 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:33,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,460 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,460 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,460 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:33,460 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,461 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:33,461 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:33,461 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:33,461 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:33,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:33,461 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-10 18:54:33,461 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:33,461 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:33,461 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:33,461 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:33,461 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:33,461 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,561 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:33,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:33,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:33,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:33,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,568 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:33,568 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:33,568 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:33,568 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:33,568 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,568 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,570 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:33,570 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:33,570 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:33,570 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,572 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:33,572 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:33,572 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:33,572 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:33,572 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:33,572 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:33,572 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:33,572 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:33,572 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. started... 2023-03-10 18:54:33,572 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b416ae27480b11a339c44b606dbd5830 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:33,985 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509076ee62ce611d54540be1671840fe109e7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201509076ee62ce611d54540be1671840fe109e7 2023-03-10 18:54:34,391 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/188b30f1b92740998edc16a6a27974ed 2023-03-10 18:54:34,396 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/188b30f1b92740998edc16a6a27974ed as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/188b30f1b92740998edc16a6a27974ed 2023-03-10 18:54:34,401 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/188b30f1b92740998edc16a6a27974ed, entries=1, sequenceid=37, filesize=4.9 K 2023-03-10 18:54:34,402 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b416ae27480b11a339c44b606dbd5830 in 830ms, sequenceid=37, compaction requested=true 2023-03-10 18:54:34,402 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:34,402 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:34,402 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:34,402 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:34,402 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,402 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:34,402 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:34,404 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,404 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:34,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:34,404 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,404 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:34,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:34,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:34,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:34,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:34,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b0fed68[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:34,405 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:34,405 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,407 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,407 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:34,407 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:34,407 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:34,407 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,407 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:34,408 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:34,409 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,409 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:34,409 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:34,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:34,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:34,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:34,411 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:34,411 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:34,411 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:34,411 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,411 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:34,414 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,414 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:34,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:34,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,414 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:34,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,414 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:34,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:34,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:34,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:34,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,415 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:34,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,417 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,417 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,417 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,417 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:34,417 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,418 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:34,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-10 18:54:34,418 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:34,418 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:34,418 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:34,418 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:34,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:34,418 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:34,418 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:34,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:34,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:34,518 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:34,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:34,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:34,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:34,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,525 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:34,525 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:34,525 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-10 18:54:34,525 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-10 18:54:34,525 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,525 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,527 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:34,527 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:34,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:34,527 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:34,527 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:34,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:34,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-10 18:54:34,528 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:34,528 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-10 18:54:34,528 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:34,528 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-10 18:54:34,529 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:34,529 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:34,529 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:34,529 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. started... 2023-03-10 18:54:34,529 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b416ae27480b11a339c44b606dbd5830 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:34,942 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120c7e0d19c967e44e0b92502e8fa86b6d8 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120c7e0d19c967e44e0b92502e8fa86b6d8 2023-03-10 18:54:35,347 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/9b22f927647e4786b8a0bfa80b0a0c88 2023-03-10 18:54:35,352 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/9b22f927647e4786b8a0bfa80b0a0c88 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/9b22f927647e4786b8a0bfa80b0a0c88 2023-03-10 18:54:35,356 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/9b22f927647e4786b8a0bfa80b0a0c88, entries=1, sequenceid=41, filesize=4.9 K 2023-03-10 18:54:35,357 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b416ae27480b11a339c44b606dbd5830 in 828ms, sequenceid=41, compaction requested=true 2023-03-10 18:54:35,357 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:35,357 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:35,358 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:35,358 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:35,358 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,358 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-10 18:54:35,358 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-10 18:54:35,359 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,359 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:35,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:35,360 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,360 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:35,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:35,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:35,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:35,361 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-10 18:54:35,361 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-10 18:54:35,361 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4383d16[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:35,361 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,364 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,364 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,364 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-10 18:54:35,364 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:35,364 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-10 18:54:35,364 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:35,366 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-10 18:54:35,366 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:35,366 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,366 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-10 18:54:35,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:35,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:35,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:35,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:35,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:35,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:35,368 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:54:35,368 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-10 18:54:35,368 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:35,368 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:35,368 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,368 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:35,369 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,369 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,369 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:35,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:35,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:35,370 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:35,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,371 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:35,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,376 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:35,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,377 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:35,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-10 18:54:35,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:35,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:35,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:35,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:35,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:35,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,377 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:35,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:35,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:35,377 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:35,377 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,478 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:35,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:35,480 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-10 18:54:35,482 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:54:35,482 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:54:35,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:35,483 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475075483 2023-03-10 18:54:35,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:54:35,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-10 18:54:35,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:35,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x0f266190 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:54:35,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@58a49206, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:54:35,491 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:36,336 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033d772ea7a08540059b7cc6ec9eab3ab4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601033d772ea7a08540059b7cc6ec9eab3ab4 2023-03-10 18:54:36,338 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:36,350 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:54:36,352 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44268, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:54:36,359 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:54:36,361 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49174, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:54:36,370 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0964ef3995d44806a2ba6d4f24e88271 first=Optional[r3] last=Optional[r6] 2023-03-10 18:54:36,371 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0964ef3995d44806a2ba6d4f24e88271}] 2023-03-10 18:54:36,375 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0964ef3995d44806a2ba6d4f24e88271 for inclusion in b416ae27480b11a339c44b606dbd5830/family1 2023-03-10 18:54:36,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-10 18:54:36,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:54:36,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:36,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0964ef3995d44806a2ba6d4f24e88271 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyPolicy__t0n5tsmh6dinrm7a84qj0kdnu9pcljcjba0bv55lekfmiihmf64vcdm30ehbpeur/family1/0964ef3995d44806a2ba6d4f24e88271 2023-03-10 18:54:36,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyPolicy__t0n5tsmh6dinrm7a84qj0kdnu9pcljcjba0bv55lekfmiihmf64vcdm30ehbpeur/family1/0964ef3995d44806a2ba6d4f24e88271 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/7bcf847b0f854c98a1521df7d19246ee_SeqId_44_ 2023-03-10 18:54:36,382 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyPolicy__t0n5tsmh6dinrm7a84qj0kdnu9pcljcjba0bv55lekfmiihmf64vcdm30ehbpeur/family1/0964ef3995d44806a2ba6d4f24e88271 into b416ae27480b11a339c44b606dbd5830/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/7bcf847b0f854c98a1521df7d19246ee_SeqId_44_ - updating store file list. 2023-03-10 18:54:36,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/7bcf847b0f854c98a1521df7d19246ee_SeqId_44_ into b416ae27480b11a339c44b606dbd5830/family1 2023-03-10 18:54:36,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyPolicy__t0n5tsmh6dinrm7a84qj0kdnu9pcljcjba0bv55lekfmiihmf64vcdm30ehbpeur/family1/0964ef3995d44806a2ba6d4f24e88271 into b416ae27480b11a339c44b606dbd5830/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/7bcf847b0f854c98a1521df7d19246ee_SeqId_44_) 2023-03-10 18:54:36,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyPolicy__t0n5tsmh6dinrm7a84qj0kdnu9pcljcjba0bv55lekfmiihmf64vcdm30ehbpeur/family1/0964ef3995d44806a2ba6d4f24e88271 2023-03-10 18:54:36,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-10 18:54:36,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-10 18:54:36,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): b416ae27480b11a339c44b606dbd5830/family1 is initiating minor compaction 2023-03-10 18:54:36,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@20930621; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:36,388 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b416ae27480b11a339c44b606dbd5830/family1 in testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:54:36,388 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/bc9ea7bb586e4688aeffe0fbe42dbe0e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/fc0bab1f2a924cd1b934b35c12d03b3c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/3d63da2188ae4482a9ef1b9339263d1b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/6ded93454b4c4991ac39da3243112db2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/25928ec7c38a42dc8b9911e2f196d39b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/2eb0b443b2d14b86a266e9369c54e508, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/e205d865714145b6a1d02ea25ff0b30e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/188b30f1b92740998edc16a6a27974ed, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/9b22f927647e4786b8a0bfa80b0a0c88, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/7bcf847b0f854c98a1521df7d19246ee_SeqId_44_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp, totalSize=50.0 K 2023-03-10 18:54:36,388 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting bc9ea7bb586e4688aeffe0fbe42dbe0e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-10 18:54:36,388 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting fc0bab1f2a924cd1b934b35c12d03b3c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-10 18:54:36,388 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 3d63da2188ae4482a9ef1b9339263d1b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-10 18:54:36,388 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 6ded93454b4c4991ac39da3243112db2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-10 18:54:36,388 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 25928ec7c38a42dc8b9911e2f196d39b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-10 18:54:36,389 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 2eb0b443b2d14b86a266e9369c54e508, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-10 18:54:36,389 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting e205d865714145b6a1d02ea25ff0b30e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-10 18:54:36,389 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 188b30f1b92740998edc16a6a27974ed, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-10 18:54:36,389 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 9b22f927647e4786b8a0bfa80b0a0c88, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-10 18:54:36,389 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 7bcf847b0f854c98a1521df7d19246ee_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-10 18:54:36,390 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:36,391 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:36,392 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122838d15f31f34b44e1a445ceef72f998a7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122838d15f31f34b44e1a445ceef72f998a7 2023-03-10 18:54:36,394 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123184af2887fc5845c6a6318c9196115937 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123184af2887fc5845c6a6318c9196115937 2023-03-10 18:54:36,395 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101eedfea0407a649f7bfbeb10d2b948b15 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101eedfea0407a649f7bfbeb10d2b948b15 2023-03-10 18:54:36,396 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103fcffa4d696354ea7a77e3920f7a94a56 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103fcffa4d696354ea7a77e3920f7a94a56 2023-03-10 18:54:36,396 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-10 18:54:36,396 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-10 18:54:36,397 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:36,397 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:36,397 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:36,397 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:36,397 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:36,397 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:36,397 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:36,397 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:36,414 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): b416ae27480b11a339c44b606dbd5830#family1#compaction#49 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:54:36,438 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512053b5130ba0b334553ac6d950848bf52a3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512053b5130ba0b334553ac6d950848bf52a3 2023-03-10 18:54:36,439 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:36,469 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e07154ea4138440cab985e45b2a9e3b5 first=Optional[r00] last=Optional[r2] 2023-03-10 18:54:36,470 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e07154ea4138440cab985e45b2a9e3b5}] 2023-03-10 18:54:36,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e07154ea4138440cab985e45b2a9e3b5 for inclusion in b416ae27480b11a339c44b606dbd5830/family1 2023-03-10 18:54:36,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-10 18:54:36,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:54:36,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:36,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e07154ea4138440cab985e45b2a9e3b5 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ipe1edjt7tacij2nopop32fk9t0idfdsq5jkm48rqkmnr5d8he7plo11154qgb50/family1/e07154ea4138440cab985e45b2a9e3b5 2023-03-10 18:54:36,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ipe1edjt7tacij2nopop32fk9t0idfdsq5jkm48rqkmnr5d8he7plo11154qgb50/family1/e07154ea4138440cab985e45b2a9e3b5 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/c43511b5c19c46e69bc7a70426ddddfc_SeqId_46_ 2023-03-10 18:54:36,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ipe1edjt7tacij2nopop32fk9t0idfdsq5jkm48rqkmnr5d8he7plo11154qgb50/family1/e07154ea4138440cab985e45b2a9e3b5 into b416ae27480b11a339c44b606dbd5830/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/c43511b5c19c46e69bc7a70426ddddfc_SeqId_46_ - updating store file list. 2023-03-10 18:54:36,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/c43511b5c19c46e69bc7a70426ddddfc_SeqId_46_ into b416ae27480b11a339c44b606dbd5830/family1 2023-03-10 18:54:36,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ipe1edjt7tacij2nopop32fk9t0idfdsq5jkm48rqkmnr5d8he7plo11154qgb50/family1/e07154ea4138440cab985e45b2a9e3b5 into b416ae27480b11a339c44b606dbd5830/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/c43511b5c19c46e69bc7a70426ddddfc_SeqId_46_) 2023-03-10 18:54:36,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ipe1edjt7tacij2nopop32fk9t0idfdsq5jkm48rqkmnr5d8he7plo11154qgb50/family1/e07154ea4138440cab985e45b2a9e3b5 2023-03-10 18:54:36,484 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:36,485 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:36,486 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511302aaeb17143c34b09be35addc5af1d567 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511302aaeb17143c34b09be35addc5af1d567 2023-03-10 18:54:36,487 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201337e3e71c59a41afaefe88c3b5065037 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201337e3e71c59a41afaefe88c3b5065037 2023-03-10 18:54:36,488 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205fd2f313567a04fa0bc7e44be9a776de0 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205fd2f313567a04fa0bc7e44be9a776de0 2023-03-10 18:54:36,488 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-10 18:54:36,488 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-10 18:54:36,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-10 18:54:36,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-10 18:54:36,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:36,490 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:54:36,492 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.0090 sec 2023-03-10 18:54:36,605 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=258 (was 251) - Thread LEAK? -, OpenFileDescriptor=634 (was 591) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=260 (was 275), ProcessCount=170 (was 170), AvailableMemoryMB=5639 (was 5618) - AvailableMemoryMB LEAK? - 2023-03-10 18:54:36,617 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=258, OpenFileDescriptor=634, MaxFileDescriptor=60000, SystemLoadAverage=260, ProcessCount=170, AvailableMemoryMB=5638 2023-03-10 18:54:36,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-10 18:54:36,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:36,622 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:54:36,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-10 18:54:36,623 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:54:36,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-10 18:54:36,624 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:36,625 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb empty. 2023-03-10 18:54:36,625 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:36,625 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-10 18:54:36,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-10 18:54:36,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-10 18:54:37,033 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-10 18:54:37,035 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => b7c0739076470c4809433d5b3c9d4beb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:54:37,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-10 18:54:37,233 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/.tmp/family1/e06c4b2dfd0d48c78875af1c21d6454f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/e06c4b2dfd0d48c78875af1c21d6454f 2023-03-10 18:54:37,238 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in b416ae27480b11a339c44b606dbd5830/family1 of b416ae27480b11a339c44b606dbd5830 into e06c4b2dfd0d48c78875af1c21d6454f(size=6.6 K), total size for store is 337.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:54:37,238 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:54:37,238 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678474476388; duration=0sec 2023-03-10 18:54:37,238 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:37,325 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/bc9ea7bb586e4688aeffe0fbe42dbe0e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/fc0bab1f2a924cd1b934b35c12d03b3c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/3d63da2188ae4482a9ef1b9339263d1b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/6ded93454b4c4991ac39da3243112db2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/25928ec7c38a42dc8b9911e2f196d39b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/2eb0b443b2d14b86a266e9369c54e508, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/e205d865714145b6a1d02ea25ff0b30e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/188b30f1b92740998edc16a6a27974ed, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/9b22f927647e4786b8a0bfa80b0a0c88, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/7bcf847b0f854c98a1521df7d19246ee_SeqId_44_] to archive 2023-03-10 18:54:37,326 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:37,328 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/bc9ea7bb586e4688aeffe0fbe42dbe0e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/bc9ea7bb586e4688aeffe0fbe42dbe0e 2023-03-10 18:54:37,330 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/fc0bab1f2a924cd1b934b35c12d03b3c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/fc0bab1f2a924cd1b934b35c12d03b3c 2023-03-10 18:54:37,331 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/3d63da2188ae4482a9ef1b9339263d1b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/3d63da2188ae4482a9ef1b9339263d1b 2023-03-10 18:54:37,332 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/6ded93454b4c4991ac39da3243112db2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/6ded93454b4c4991ac39da3243112db2 2023-03-10 18:54:37,333 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/25928ec7c38a42dc8b9911e2f196d39b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/25928ec7c38a42dc8b9911e2f196d39b 2023-03-10 18:54:37,334 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/2eb0b443b2d14b86a266e9369c54e508 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/2eb0b443b2d14b86a266e9369c54e508 2023-03-10 18:54:37,335 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/e205d865714145b6a1d02ea25ff0b30e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/e205d865714145b6a1d02ea25ff0b30e 2023-03-10 18:54:37,336 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/188b30f1b92740998edc16a6a27974ed to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/188b30f1b92740998edc16a6a27974ed 2023-03-10 18:54:37,337 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/9b22f927647e4786b8a0bfa80b0a0c88 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/9b22f927647e4786b8a0bfa80b0a0c88 2023-03-10 18:54:37,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/7bcf847b0f854c98a1521df7d19246ee_SeqId_44_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/family1/7bcf847b0f854c98a1521df7d19246ee_SeqId_44_ 2023-03-10 18:54:37,441 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:37,441 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing b7c0739076470c4809433d5b3c9d4beb, disabling compactions & flushes 2023-03-10 18:54:37,441 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:37,441 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:37,441 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. after waiting 0 ms 2023-03-10 18:54:37,441 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:37,441 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:37,441 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:37,444 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:54:37,445 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678474477445"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474477445"}]},"ts":"1678474477445"} 2023-03-10 18:54:37,446 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-10 18:54:37,447 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:54:37,447 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474477447"}]},"ts":"1678474477447"} 2023-03-10 18:54:37,448 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-10 18:54:37,452 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, ASSIGN}] 2023-03-10 18:54:37,454 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, ASSIGN 2023-03-10 18:54:37,455 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:54:37,606 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=b7c0739076470c4809433d5b3c9d4beb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:37,606 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678474477606"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474477606"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474477606"}]},"ts":"1678474477606"} 2023-03-10 18:54:37,608 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure b7c0739076470c4809433d5b3c9d4beb, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:54:37,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-10 18:54:37,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:37,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b7c0739076470c4809433d5b3c9d4beb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:54:37,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:37,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:37,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:37,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:37,765 INFO [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:37,766 DEBUG [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:37,766 DEBUG [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:37,767 INFO [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b7c0739076470c4809433d5b3c9d4beb columnFamilyName family1 2023-03-10 18:54:37,768 INFO [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] regionserver.HStore(310): Store=b7c0739076470c4809433d5b3c9d4beb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:37,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:37,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:37,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:37,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:54:37,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b7c0739076470c4809433d5b3c9d4beb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74758649, jitterRate=0.1139906793832779}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:54:37,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:37,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb., pid=41, masterSystemTime=1678474477759 2023-03-10 18:54:37,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:37,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:37,777 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=b7c0739076470c4809433d5b3c9d4beb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:37,777 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678474477777"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474477777"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474477777"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474477777"}]},"ts":"1678474477777"} 2023-03-10 18:54:37,779 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-10 18:54:37,779 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure b7c0739076470c4809433d5b3c9d4beb, server=jenkins-hbase4.apache.org,43181,1678474414959 in 170 msec 2023-03-10 18:54:37,781 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-10 18:54:37,781 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, ASSIGN in 327 msec 2023-03-10 18:54:37,782 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:54:37,782 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474477782"}]},"ts":"1678474477782"} 2023-03-10 18:54:37,783 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-10 18:54:37,786 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:54:37,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1660 sec 2023-03-10 18:54:38,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-10 18:54:38,729 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-10 18:54:38,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:38,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:38,761 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:38,761 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:38,761 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-10 18:54:38,761 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-10 18:54:38,761 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:38,761 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:38,763 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:38,763 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:38,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:38,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:38,763 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:38,763 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:38,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:38,764 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:38,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:38,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:38,764 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:38,764 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:38,764 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-10 18:54:38,765 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:38,765 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-10 18:54:38,765 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:38,765 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:38,765 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:38,765 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. started... 2023-03-10 18:54:38,766 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b7c0739076470c4809433d5b3c9d4beb 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-10 18:54:38,791 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113018b91ebf04f04cfa97dad928e80a6576 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113018b91ebf04f04cfa97dad928e80a6576 2023-03-10 18:54:39,195 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/521abc800cc04ca1943b7ae4cea524a9 2023-03-10 18:54:39,200 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/521abc800cc04ca1943b7ae4cea524a9 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/521abc800cc04ca1943b7ae4cea524a9 2023-03-10 18:54:39,204 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/521abc800cc04ca1943b7ae4cea524a9, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-10 18:54:39,205 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for b7c0739076470c4809433d5b3c9d4beb in 440ms, sequenceid=5, compaction requested=false 2023-03-10 18:54:39,205 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:39,206 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:39,206 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-10 18:54:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-10 18:54:39,209 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,209 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:39,209 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,209 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:39,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-10 18:54:39,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7765fe9a[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:39,211 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-10 18:54:39,211 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,212 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,212 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-10 18:54:39,212 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-10 18:54:39,212 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:39,212 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,212 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:39,213 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-10 18:54:39,213 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,213 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:39,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,214 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-10 18:54:39,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:39,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:39,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:39,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:39,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:39,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:39,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-10 18:54:39,215 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-10 18:54:39,216 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:39,216 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:39,216 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,216 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:39,217 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,217 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,217 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:39,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:39,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:39,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:39,218 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:39,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,219 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:39,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,222 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:39,222 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:39,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:39,222 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:39,222 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:39,222 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,222 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:39,222 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:39,222 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:39,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:39,223 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,223 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,223 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,223 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,223 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:39,223 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:39,323 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:39,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:39,328 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:39,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:39,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,330 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:39,330 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:39,330 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-10 18:54:39,330 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-10 18:54:39,330 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,330 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,332 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:39,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:39,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:39,332 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,332 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:39,332 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:39,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:39,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,333 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,333 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-10 18:54:39,333 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:39,333 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-10 18:54:39,333 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:39,333 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-10 18:54:39,333 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:39,333 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:39,333 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:39,333 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. started... 2023-03-10 18:54:39,334 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b7c0739076470c4809433d5b3c9d4beb 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-10 18:54:39,747 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511280b5334e0694546c2acc83330dc6647e9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511280b5334e0694546c2acc83330dc6647e9 2023-03-10 18:54:40,154 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/2092b36aefbf42818ee276785fb09c97 2023-03-10 18:54:40,159 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/2092b36aefbf42818ee276785fb09c97 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2092b36aefbf42818ee276785fb09c97 2023-03-10 18:54:40,163 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2092b36aefbf42818ee276785fb09c97, entries=1, sequenceid=9, filesize=5.0 K 2023-03-10 18:54:40,164 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for b7c0739076470c4809433d5b3c9d4beb in 830ms, sequenceid=9, compaction requested=false 2023-03-10 18:54:40,164 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:40,164 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:40,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:40,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:40,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-10 18:54:40,165 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-10 18:54:40,166 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,166 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:40,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:40,166 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,167 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:40,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:40,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:40,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:40,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-10 18:54:40,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24a754fd[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:40,168 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-10 18:54:40,168 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,169 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,169 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-10 18:54:40,169 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-10 18:54:40,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:40,170 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,170 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:40,171 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,171 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-10 18:54:40,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:40,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:40,171 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:40,171 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-10 18:54:40,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:40,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:40,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:40,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-10 18:54:40,174 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-10 18:54:40,174 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:40,174 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:40,174 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,174 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:40,176 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,176 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,176 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:40,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:40,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:40,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:40,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:40,177 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:40,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:40,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,178 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,183 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,183 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:40,183 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:40,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:40,183 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,183 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:40,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:40,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:40,183 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:40,183 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:40,183 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:40,183 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:40,184 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:40,184 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,184 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,284 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:40,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:40,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:40,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:40,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,290 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:40,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:40,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-10 18:54:40,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-10 18:54:40,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,292 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:40,292 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:40,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:40,292 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:40,292 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:40,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:40,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-10 18:54:40,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:40,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-10 18:54:40,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:40,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-10 18:54:40,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:40,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:40,294 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:40,294 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. started... 2023-03-10 18:54:40,294 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b7c0739076470c4809433d5b3c9d4beb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:40,698 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:54:40,698 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0f266190 to 127.0.0.1:56431 2023-03-10 18:54:40,698 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:54:40,699 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:54:40,699 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x63825d30 to 127.0.0.1:56431 2023-03-10 18:54:40,699 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:54:40,699 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:54:40,699 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x56649942 to 127.0.0.1:56431 2023-03-10 18:54:40,699 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:54:40,709 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201882d1153c569457da6300957676b6c86 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201882d1153c569457da6300957676b6c86 2023-03-10 18:54:41,115 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/133fcb102a8f409a9c27e8dc1a406ecf 2023-03-10 18:54:41,122 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/133fcb102a8f409a9c27e8dc1a406ecf as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/133fcb102a8f409a9c27e8dc1a406ecf 2023-03-10 18:54:41,126 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/133fcb102a8f409a9c27e8dc1a406ecf, entries=1, sequenceid=13, filesize=5.0 K 2023-03-10 18:54:41,127 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7c0739076470c4809433d5b3c9d4beb in 833ms, sequenceid=13, compaction requested=true 2023-03-10 18:54:41,127 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:41,127 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:41,127 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:41,127 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:41,127 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,127 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-10 18:54:41,127 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-10 18:54:41,130 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,130 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:41,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:41,130 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,130 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:41,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:41,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:41,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:41,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-10 18:54:41,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6799ac[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:41,132 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-10 18:54:41,132 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,133 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,133 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-10 18:54:41,133 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-10 18:54:41,133 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:41,133 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,133 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:41,135 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,135 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-10 18:54:41,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,135 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:41,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:41,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:41,135 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-10 18:54:41,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:41,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:41,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:41,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:41,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-10 18:54:41,139 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-10 18:54:41,140 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:41,140 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:41,140 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,140 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:41,141 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,141 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:41,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,141 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:41,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:41,142 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:41,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,143 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:41,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,146 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,146 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:41,146 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:41,146 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:41,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:41,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:41,146 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,146 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:41,146 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:41,147 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,147 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,147 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,147 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:41,147 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:41,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:41,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:41,247 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:41,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:41,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:41,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:41,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,254 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:41,254 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:41,254 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-10 18:54:41,254 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-10 18:54:41,254 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,254 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,255 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:41,255 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:41,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:41,256 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,256 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,256 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,256 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,256 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-10 18:54:41,257 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,257 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-10 18:54:41,257 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:41,257 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-10 18:54:41,257 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:41,257 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:41,257 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:41,257 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. started... 2023-03-10 18:54:41,257 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b7c0739076470c4809433d5b3c9d4beb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:41,269 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205aeecc20222fa4cddb2dc260e4fac0bb8 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205aeecc20222fa4cddb2dc260e4fac0bb8 2023-03-10 18:54:41,674 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/02856f001b7f4af7bb26882193acd579 2023-03-10 18:54:41,679 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/02856f001b7f4af7bb26882193acd579 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/02856f001b7f4af7bb26882193acd579 2023-03-10 18:54:41,684 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/02856f001b7f4af7bb26882193acd579, entries=1, sequenceid=17, filesize=5.0 K 2023-03-10 18:54:41,684 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7c0739076470c4809433d5b3c9d4beb in 427ms, sequenceid=17, compaction requested=true 2023-03-10 18:54:41,685 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:41,685 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:41,685 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:41,685 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:41,685 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,685 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-10 18:54:41,685 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-10 18:54:41,687 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,687 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:41,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:41,687 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,687 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:41,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:41,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:41,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:41,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-10 18:54:41,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45f2d43b[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:41,688 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-10 18:54:41,689 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,690 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,690 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-10 18:54:41,690 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-10 18:54:41,690 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,690 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:41,690 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:41,692 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-10 18:54:41,692 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,693 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:41,693 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-10 18:54:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:41,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:41,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-10 18:54:41,695 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-10 18:54:41,695 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:41,695 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:41,695 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,695 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:41,696 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,696 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,696 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:41,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:41,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:41,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:41,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:41,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:41,697 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:41,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,698 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:41,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,700 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,700 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,700 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,700 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,700 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:41,700 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:41,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:41,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-10 18:54:41,700 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:41,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:41,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:41,700 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:41,700 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,700 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,701 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:41,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:41,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:41,701 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:41,801 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:41,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:41,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:41,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:41,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,807 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:41,807 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:41,807 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-10 18:54:41,807 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-10 18:54:41,808 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,808 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,809 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,809 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:41,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:41,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:41,809 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:41,809 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:41,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,810 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:41,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,810 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,811 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-10 18:54:41,811 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-10 18:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-10 18:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:41,811 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:41,811 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. started... 2023-03-10 18:54:41,811 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b7c0739076470c4809433d5b3c9d4beb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:42,227 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228e7ef56b509764429880252f7f73efda9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228e7ef56b509764429880252f7f73efda9 2023-03-10 18:54:42,632 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/eec980d102e440d89c6c3ad90cbcb03f 2023-03-10 18:54:42,637 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/eec980d102e440d89c6c3ad90cbcb03f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/eec980d102e440d89c6c3ad90cbcb03f 2023-03-10 18:54:42,641 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/eec980d102e440d89c6c3ad90cbcb03f, entries=1, sequenceid=21, filesize=5.0 K 2023-03-10 18:54:42,642 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7c0739076470c4809433d5b3c9d4beb in 831ms, sequenceid=21, compaction requested=true 2023-03-10 18:54:42,642 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:42,642 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:42,642 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:42,642 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:42,642 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,642 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-10 18:54:42,642 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-10 18:54:42,647 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,647 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:42,648 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,648 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:42,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-10 18:54:42,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1dbb7331[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:42,649 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-10 18:54:42,649 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,650 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,650 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-10 18:54:42,650 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,650 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:42,650 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-10 18:54:42,650 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:42,652 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-10 18:54:42,652 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:42,652 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,652 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-10 18:54:42,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:42,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:42,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:42,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:42,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:42,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:42,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-10 18:54:42,654 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-10 18:54:42,654 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:42,654 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:42,654 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,654 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:42,656 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,656 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,656 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:42,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:42,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:42,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:42,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:42,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:42,657 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:42,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:42,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,659 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:42,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:42,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,661 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:42,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:42,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:42,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:42,662 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:42,661 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,662 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,662 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:42,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:42,662 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:42,662 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:42,762 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:42,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:42,767 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:42,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:42,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,769 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:42,770 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:42,770 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-10 18:54:42,770 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-10 18:54:42,770 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,770 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,772 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,772 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:42,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:42,772 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:42,772 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,772 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,773 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,773 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-10 18:54:42,773 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:42,773 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-10 18:54:42,773 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:42,773 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-10 18:54:42,773 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:42,773 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:42,773 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:42,774 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. started... 2023-03-10 18:54:42,774 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b7c0739076470c4809433d5b3c9d4beb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:43,190 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316de99a2ff9ff44a19b1b5566423a7209 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512316de99a2ff9ff44a19b1b5566423a7209 2023-03-10 18:54:43,195 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/d3e2050eb7614f9d822080af47aed2ee 2023-03-10 18:54:43,199 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/d3e2050eb7614f9d822080af47aed2ee as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d3e2050eb7614f9d822080af47aed2ee 2023-03-10 18:54:43,203 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d3e2050eb7614f9d822080af47aed2ee, entries=1, sequenceid=25, filesize=5.0 K 2023-03-10 18:54:43,204 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7c0739076470c4809433d5b3c9d4beb in 430ms, sequenceid=25, compaction requested=true 2023-03-10 18:54:43,204 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:43,204 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:43,204 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:43,204 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:43,204 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,204 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-10 18:54:43,204 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-10 18:54:43,206 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,206 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:43,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:43,206 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,206 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:43,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:43,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:43,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:43,208 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-10 18:54:43,208 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a5ccc7[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:43,208 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-10 18:54:43,208 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,209 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,210 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-10 18:54:43,210 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-10 18:54:43,210 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:43,210 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,210 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:43,212 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-10 18:54:43,212 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:43,212 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-10 18:54:43,212 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:43,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:43,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:43,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:43,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:43,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:43,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-10 18:54:43,214 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-10 18:54:43,215 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:43,215 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:43,215 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,215 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:43,216 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,216 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:43,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:43,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,216 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:43,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:43,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:43,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:43,217 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,218 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,220 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,220 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:43,220 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:43,220 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:43,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:43,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-10 18:54:43,220 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:43,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:43,220 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:43,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:43,220 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:43,221 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,221 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,221 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,221 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:43,221 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:43,301 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:54:43,321 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:43,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:43,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:43,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:43,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,328 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:43,328 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:43,328 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-10 18:54:43,328 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-10 18:54:43,329 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,329 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,330 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,330 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:43,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:43,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:43,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,331 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:43,331 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:43,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:43,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-10 18:54:43,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:43,332 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-10 18:54:43,332 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:43,332 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-10 18:54:43,336 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:43,336 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:43,336 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:43,336 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. started... 2023-03-10 18:54:43,336 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b7c0739076470c4809433d5b3c9d4beb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:43,749 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101ec99e8a645ed4c99aaf921411ed4e116 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101ec99e8a645ed4c99aaf921411ed4e116 2023-03-10 18:54:44,154 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/93d9073993d5413d811889c29ab8eb6e 2023-03-10 18:54:44,159 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/93d9073993d5413d811889c29ab8eb6e as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/93d9073993d5413d811889c29ab8eb6e 2023-03-10 18:54:44,163 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/93d9073993d5413d811889c29ab8eb6e, entries=1, sequenceid=29, filesize=5.0 K 2023-03-10 18:54:44,164 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7c0739076470c4809433d5b3c9d4beb in 828ms, sequenceid=29, compaction requested=true 2023-03-10 18:54:44,164 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:44,164 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:44,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:44,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:44,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-10 18:54:44,164 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-10 18:54:44,167 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,167 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:44,167 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,167 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:44,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:44,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-10 18:54:44,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3fb39fe6[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:44,168 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-10 18:54:44,168 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,170 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-10 18:54:44,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-10 18:54:44,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:44,170 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,170 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:44,171 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-10 18:54:44,171 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,171 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:44,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,171 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-10 18:54:44,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:44,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:44,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:44,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:44,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:44,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-10 18:54:44,173 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-10 18:54:44,174 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:44,174 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:44,174 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,174 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:44,179 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,179 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,179 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:44,179 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,180 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:44,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:44,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:44,182 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:44,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:44,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:44,182 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:44,183 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-10 18:54:44,183 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:44,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:44,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:44,183 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:44,283 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:44,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:44,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:44,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:44,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,290 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:44,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:44,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-10 18:54:44,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-10 18:54:44,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,290 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,292 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:44,292 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:44,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:44,292 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:44,292 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:44,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:44,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-10 18:54:44,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:44,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-10 18:54:44,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:44,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-10 18:54:44,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:44,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:44,293 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:44,293 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. started... 2023-03-10 18:54:44,293 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b7c0739076470c4809433d5b3c9d4beb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:44,707 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103e99056200b5c41d3968687d0c83c0a23 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103e99056200b5c41d3968687d0c83c0a23 2023-03-10 18:54:45,113 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/d72a20cd0c9b47a48fb5368eb32f9d59 2023-03-10 18:54:45,118 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/d72a20cd0c9b47a48fb5368eb32f9d59 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d72a20cd0c9b47a48fb5368eb32f9d59 2023-03-10 18:54:45,121 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d72a20cd0c9b47a48fb5368eb32f9d59, entries=1, sequenceid=33, filesize=5.0 K 2023-03-10 18:54:45,122 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7c0739076470c4809433d5b3c9d4beb in 829ms, sequenceid=33, compaction requested=true 2023-03-10 18:54:45,122 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:45,122 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:45,122 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:45,122 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:45,122 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,122 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-10 18:54:45,122 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-10 18:54:45,124 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,124 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:45,125 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,125 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:45,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-10 18:54:45,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6cb3d727[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:45,126 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-10 18:54:45,126 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,127 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,127 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-10 18:54:45,127 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-10 18:54:45,127 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:45,128 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,128 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:45,129 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-10 18:54:45,129 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,129 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:45,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,129 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-10 18:54:45,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:45,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:45,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:45,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:45,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:45,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:45,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-10 18:54:45,131 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-10 18:54:45,131 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:45,131 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:45,131 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,131 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:45,133 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,133 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:45,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,133 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:45,134 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:45,135 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,137 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,137 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:45,137 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,137 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:45,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:45,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:45,137 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:45,137 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:45,137 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-10 18:54:45,137 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:45,137 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:45,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:45,137 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,137 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,137 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,237 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:45,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:45,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:45,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:45,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,244 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:45,244 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:45,244 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-10 18:54:45,244 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-10 18:54:45,244 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,244 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,246 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:45,246 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:45,246 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:45,246 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,247 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:45,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,247 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,247 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-10 18:54:45,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:45,247 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-10 18:54:45,247 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:45,247 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-10 18:54:45,247 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:45,247 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:45,247 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:45,248 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. started... 2023-03-10 18:54:45,248 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b7c0739076470c4809433d5b3c9d4beb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:45,662 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509072a8c4a86f343465db3bb2bbb96ed08aa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201509072a8c4a86f343465db3bb2bbb96ed08aa 2023-03-10 18:54:46,068 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/79a08538d4484afb854c007ac84e6bdd 2023-03-10 18:54:46,072 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/79a08538d4484afb854c007ac84e6bdd as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/79a08538d4484afb854c007ac84e6bdd 2023-03-10 18:54:46,076 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/79a08538d4484afb854c007ac84e6bdd, entries=1, sequenceid=37, filesize=5.0 K 2023-03-10 18:54:46,077 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7c0739076470c4809433d5b3c9d4beb in 829ms, sequenceid=37, compaction requested=true 2023-03-10 18:54:46,077 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:46,077 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:46,077 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:46,077 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:46,077 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,077 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-10 18:54:46,077 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-10 18:54:46,079 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,079 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:46,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:46,079 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,079 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:46,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:46,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:46,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:46,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-10 18:54:46,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a57137b[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:46,080 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-10 18:54:46,081 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,083 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,083 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-10 18:54:46,083 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-10 18:54:46,083 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:46,083 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,083 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:46,085 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-10 18:54:46,085 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:46,085 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,085 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-10 18:54:46,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:46,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:46,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:46,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:46,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:46,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:46,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-10 18:54:46,087 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-10 18:54:46,087 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:46,087 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:46,087 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,087 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:46,088 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,088 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:46,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,088 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:46,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:46,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:46,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:46,089 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:46,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:46,090 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:46,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,095 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:46,095 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:46,095 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,096 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:46,096 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:46,095 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:46,096 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,096 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:46,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:46,096 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,096 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:46,096 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-10 18:54:46,096 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:46,096 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:46,096 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:46,096 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,196 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:46,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:46,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:46,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:46,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,203 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:46,203 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:46,203 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-10 18:54:46,203 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-10 18:54:46,203 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,203 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,205 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:46,205 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:46,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:46,205 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:46,205 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:46,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,206 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:46,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,206 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,206 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-10 18:54:46,207 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-10 18:54:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-10 18:54:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:46,207 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:46,207 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. started... 2023-03-10 18:54:46,207 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b7c0739076470c4809433d5b3c9d4beb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:46,621 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120de8e7a3004d440d5a4aae76fe22ccee9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120de8e7a3004d440d5a4aae76fe22ccee9 2023-03-10 18:54:47,027 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/45d03717ed9b455d92683614fde34588 2023-03-10 18:54:47,032 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/45d03717ed9b455d92683614fde34588 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/45d03717ed9b455d92683614fde34588 2023-03-10 18:54:47,035 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/45d03717ed9b455d92683614fde34588, entries=1, sequenceid=41, filesize=5.0 K 2023-03-10 18:54:47,036 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7c0739076470c4809433d5b3c9d4beb in 829ms, sequenceid=41, compaction requested=true 2023-03-10 18:54:47,036 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:47,036 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-10 18:54:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-10 18:54:47,038 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,038 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:47,039 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,039 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:47,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-10 18:54:47,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b92102[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:47,040 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-10 18:54:47,040 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,041 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,041 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-10 18:54:47,041 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-10 18:54:47,041 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:47,041 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,042 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-10 18:54:47,044 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-10 18:54:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:47,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:47,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-10 18:54:47,046 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-10 18:54:47,046 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:47,046 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:47,046 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,046 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:47,048 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,048 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:47,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:47,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,048 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:47,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:47,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:47,049 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,050 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,052 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:47,052 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:47,052 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:47,052 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:47,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-10 18:54:47,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:47,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:47,052 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,052 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,052 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:47,052 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,052 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,052 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,053 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:47,053 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:47,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:47,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:47,153 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:47,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:47,155 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:47,156 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-10 18:54:47,157 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-10 18:54:47,158 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:47,158 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475087158 2023-03-10 18:54:47,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:54:47,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-10 18:54:47,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:47,160 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x5164c13c to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:54:47,164 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2dac4db4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:54:47,165 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:48,003 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035a18c30ba8e64987ab8d3d4aaa97c244 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601035a18c30ba8e64987ab8d3d4aaa97c244 2023-03-10 18:54:48,004 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:48,016 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:54:48,017 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58966, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:54:48,023 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:54:48,025 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33408, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:54:48,037 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/814fb6cfb226463fb527a29c27a3328b first=Optional[r3] last=Optional[r6] 2023-03-10 18:54:48,038 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/814fb6cfb226463fb527a29c27a3328b}] 2023-03-10 18:54:48,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/814fb6cfb226463fb527a29c27a3328b for inclusion in b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:48,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-10 18:54:48,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:54:48,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:48,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/814fb6cfb226463fb527a29c27a3328b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__histq6gt3ndsl445rvijivebt4faljoq8v8rl2qtb92pjgrrleeprrksjcbjmkon/family1/814fb6cfb226463fb527a29c27a3328b 2023-03-10 18:54:48,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__histq6gt3ndsl445rvijivebt4faljoq8v8rl2qtb92pjgrrleeprrksjcbjmkon/family1/814fb6cfb226463fb527a29c27a3328b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/bfce4f738b8d44409386ba59d150c3be_SeqId_44_ 2023-03-10 18:54:48,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__histq6gt3ndsl445rvijivebt4faljoq8v8rl2qtb92pjgrrleeprrksjcbjmkon/family1/814fb6cfb226463fb527a29c27a3328b into b7c0739076470c4809433d5b3c9d4beb/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/bfce4f738b8d44409386ba59d150c3be_SeqId_44_ - updating store file list. 2023-03-10 18:54:48,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/bfce4f738b8d44409386ba59d150c3be_SeqId_44_ into b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:48,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__histq6gt3ndsl445rvijivebt4faljoq8v8rl2qtb92pjgrrleeprrksjcbjmkon/family1/814fb6cfb226463fb527a29c27a3328b into b7c0739076470c4809433d5b3c9d4beb/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/bfce4f738b8d44409386ba59d150c3be_SeqId_44_) 2023-03-10 18:54:48,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__histq6gt3ndsl445rvijivebt4faljoq8v8rl2qtb92pjgrrleeprrksjcbjmkon/family1/814fb6cfb226463fb527a29c27a3328b 2023-03-10 18:54:48,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-10 18:54:48,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-10 18:54:48,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): b7c0739076470c4809433d5b3c9d4beb/family1 is initiating minor compaction 2023-03-10 18:54:48,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@18136906; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:48,056 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b7c0739076470c4809433d5b3c9d4beb/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:48,056 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2092b36aefbf42818ee276785fb09c97, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/133fcb102a8f409a9c27e8dc1a406ecf, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/02856f001b7f4af7bb26882193acd579, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/eec980d102e440d89c6c3ad90cbcb03f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d3e2050eb7614f9d822080af47aed2ee, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/93d9073993d5413d811889c29ab8eb6e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d72a20cd0c9b47a48fb5368eb32f9d59, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/79a08538d4484afb854c007ac84e6bdd, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/45d03717ed9b455d92683614fde34588, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/bfce4f738b8d44409386ba59d150c3be_SeqId_44_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp, totalSize=50.4 K 2023-03-10 18:54:48,056 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 2092b36aefbf42818ee276785fb09c97, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-10 18:54:48,056 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 133fcb102a8f409a9c27e8dc1a406ecf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-10 18:54:48,056 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 02856f001b7f4af7bb26882193acd579, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-10 18:54:48,056 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting eec980d102e440d89c6c3ad90cbcb03f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-10 18:54:48,056 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting d3e2050eb7614f9d822080af47aed2ee, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-10 18:54:48,056 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 93d9073993d5413d811889c29ab8eb6e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-10 18:54:48,056 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting d72a20cd0c9b47a48fb5368eb32f9d59, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-10 18:54:48,056 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 79a08538d4484afb854c007ac84e6bdd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-10 18:54:48,057 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 45d03717ed9b455d92683614fde34588, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-10 18:54:48,057 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting bfce4f738b8d44409386ba59d150c3be_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-10 18:54:48,058 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:48,059 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:48,060 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228e7ef56b509764429880252f7f73efda9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228e7ef56b509764429880252f7f73efda9 2023-03-10 18:54:48,062 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512316de99a2ff9ff44a19b1b5566423a7209 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512316de99a2ff9ff44a19b1b5566423a7209 2023-03-10 18:54:48,063 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101ec99e8a645ed4c99aaf921411ed4e116 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101ec99e8a645ed4c99aaf921411ed4e116 2023-03-10 18:54:48,064 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103e99056200b5c41d3968687d0c83c0a23 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103e99056200b5c41d3968687d0c83c0a23 2023-03-10 18:54:48,064 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-10 18:54:48,064 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-10 18:54:48,065 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:48,065 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:48,065 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:48,065 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:48,065 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:48,065 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:48,065 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:48,065 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:48,085 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): b7c0739076470c4809433d5b3c9d4beb#family1#compaction#60 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 84.62 MB/second 2023-03-10 18:54:48,509 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512053aeee3cc44cb485aaab439d31db5757d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512053aeee3cc44cb485aaab439d31db5757d 2023-03-10 18:54:48,510 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:48,534 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a705afff6bb040da8cccccbf8dce5e9c first=Optional[r00] last=Optional[r2] 2023-03-10 18:54:48,535 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a705afff6bb040da8cccccbf8dce5e9c}] 2023-03-10 18:54:48,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a705afff6bb040da8cccccbf8dce5e9c for inclusion in b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:48,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-10 18:54:48,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:54:48,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:48,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a705afff6bb040da8cccccbf8dce5e9c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__laifmu3ov6cmfl95s3f6nv2eoutvpu8t5iisevcis1ro1ft7bhsu8s423lgrp1mk/family1/a705afff6bb040da8cccccbf8dce5e9c 2023-03-10 18:54:48,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__laifmu3ov6cmfl95s3f6nv2eoutvpu8t5iisevcis1ro1ft7bhsu8s423lgrp1mk/family1/a705afff6bb040da8cccccbf8dce5e9c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2a65cdbd04184d098eb9b35891747ea1_SeqId_46_ 2023-03-10 18:54:48,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__laifmu3ov6cmfl95s3f6nv2eoutvpu8t5iisevcis1ro1ft7bhsu8s423lgrp1mk/family1/a705afff6bb040da8cccccbf8dce5e9c into b7c0739076470c4809433d5b3c9d4beb/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2a65cdbd04184d098eb9b35891747ea1_SeqId_46_ - updating store file list. 2023-03-10 18:54:48,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2a65cdbd04184d098eb9b35891747ea1_SeqId_46_ into b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:48,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__laifmu3ov6cmfl95s3f6nv2eoutvpu8t5iisevcis1ro1ft7bhsu8s423lgrp1mk/family1/a705afff6bb040da8cccccbf8dce5e9c into b7c0739076470c4809433d5b3c9d4beb/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2a65cdbd04184d098eb9b35891747ea1_SeqId_46_) 2023-03-10 18:54:48,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__laifmu3ov6cmfl95s3f6nv2eoutvpu8t5iisevcis1ro1ft7bhsu8s423lgrp1mk/family1/a705afff6bb040da8cccccbf8dce5e9c 2023-03-10 18:54:48,550 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:48,550 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:48,552 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113018b91ebf04f04cfa97dad928e80a6576 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113018b91ebf04f04cfa97dad928e80a6576 2023-03-10 18:54:48,553 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201882d1153c569457da6300957676b6c86 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201882d1153c569457da6300957676b6c86 2023-03-10 18:54:48,554 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205aeecc20222fa4cddb2dc260e4fac0bb8 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205aeecc20222fa4cddb2dc260e4fac0bb8 2023-03-10 18:54:48,554 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-10 18:54:48,554 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-10 18:54:48,555 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:48,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:48,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:48,556 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-10 18:54:48,557 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.4000 sec 2023-03-10 18:54:48,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-10 18:54:48,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:48,907 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/3907738bdaff4aefac572ff9e29f17c5 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/3907738bdaff4aefac572ff9e29f17c5 2023-03-10 18:54:48,912 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in b7c0739076470c4809433d5b3c9d4beb/family1 of b7c0739076470c4809433d5b3c9d4beb into 3907738bdaff4aefac572ff9e29f17c5(size=6.8 K), total size for store is 390.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:54:48,912 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:48,912 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678474488056; duration=0sec 2023-03-10 18:54:48,912 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:48,922 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2092b36aefbf42818ee276785fb09c97, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/133fcb102a8f409a9c27e8dc1a406ecf, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/02856f001b7f4af7bb26882193acd579, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/eec980d102e440d89c6c3ad90cbcb03f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d3e2050eb7614f9d822080af47aed2ee, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/93d9073993d5413d811889c29ab8eb6e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d72a20cd0c9b47a48fb5368eb32f9d59, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/79a08538d4484afb854c007ac84e6bdd, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/45d03717ed9b455d92683614fde34588, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/bfce4f738b8d44409386ba59d150c3be_SeqId_44_] to archive 2023-03-10 18:54:48,923 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:48,925 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2092b36aefbf42818ee276785fb09c97 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2092b36aefbf42818ee276785fb09c97 2023-03-10 18:54:48,926 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/133fcb102a8f409a9c27e8dc1a406ecf to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/133fcb102a8f409a9c27e8dc1a406ecf 2023-03-10 18:54:48,928 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/02856f001b7f4af7bb26882193acd579 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/02856f001b7f4af7bb26882193acd579 2023-03-10 18:54:48,929 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/eec980d102e440d89c6c3ad90cbcb03f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/eec980d102e440d89c6c3ad90cbcb03f 2023-03-10 18:54:48,930 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d3e2050eb7614f9d822080af47aed2ee to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d3e2050eb7614f9d822080af47aed2ee 2023-03-10 18:54:48,931 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/93d9073993d5413d811889c29ab8eb6e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/93d9073993d5413d811889c29ab8eb6e 2023-03-10 18:54:48,932 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d72a20cd0c9b47a48fb5368eb32f9d59 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/d72a20cd0c9b47a48fb5368eb32f9d59 2023-03-10 18:54:48,933 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/79a08538d4484afb854c007ac84e6bdd to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/79a08538d4484afb854c007ac84e6bdd 2023-03-10 18:54:48,934 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/45d03717ed9b455d92683614fde34588 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/45d03717ed9b455d92683614fde34588 2023-03-10 18:54:48,934 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/bfce4f738b8d44409386ba59d150c3be_SeqId_44_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/bfce4f738b8d44409386ba59d150c3be_SeqId_44_ 2023-03-10 18:54:49,056 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-10 18:54:49,057 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-10 18:54:49,057 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-10 18:54:49,059 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-10 18:54:49,062 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, REOPEN/MOVE}] 2023-03-10 18:54:49,063 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, REOPEN/MOVE 2023-03-10 18:54:49,063 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b7c0739076470c4809433d5b3c9d4beb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:49,064 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678474489063"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474489063"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474489063"}]},"ts":"1678474489063"} 2023-03-10 18:54:49,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure b7c0739076470c4809433d5b3c9d4beb, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:54:49,217 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:49,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b7c0739076470c4809433d5b3c9d4beb, disabling compactions & flushes 2023-03-10 18:54:49,217 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:49,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:49,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. after waiting 0 ms 2023-03-10 18:54:49,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:49,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-10 18:54:49,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:49,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:49,223 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: b7c0739076470c4809433d5b3c9d4beb to self. 2023-03-10 18:54:49,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:49,225 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b7c0739076470c4809433d5b3c9d4beb, regionState=CLOSED 2023-03-10 18:54:49,225 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678474489225"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474489225"}]},"ts":"1678474489225"} 2023-03-10 18:54:49,227 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-10 18:54:49,227 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b7c0739076470c4809433d5b3c9d4beb, server=jenkins-hbase4.apache.org,43181,1678474414959 in 161 msec 2023-03-10 18:54:49,228 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=true 2023-03-10 18:54:49,378 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b7c0739076470c4809433d5b3c9d4beb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:49,379 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678474489378"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474489378"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474489378"}]},"ts":"1678474489378"} 2023-03-10 18:54:49,380 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure b7c0739076470c4809433d5b3c9d4beb, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:54:49,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:49,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b7c0739076470c4809433d5b3c9d4beb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:54:49,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:49,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:49,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:49,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:49,536 INFO [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:49,537 DEBUG [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:49,537 DEBUG [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:49,538 INFO [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b7c0739076470c4809433d5b3c9d4beb columnFamilyName family1 2023-03-10 18:54:49,543 DEBUG [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2a65cdbd04184d098eb9b35891747ea1_SeqId_46_ 2023-03-10 18:54:49,547 DEBUG [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/3907738bdaff4aefac572ff9e29f17c5 2023-03-10 18:54:49,550 DEBUG [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/521abc800cc04ca1943b7ae4cea524a9 2023-03-10 18:54:49,550 INFO [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] regionserver.HStore(310): Store=b7c0739076470c4809433d5b3c9d4beb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:49,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:49,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:49,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:49,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b7c0739076470c4809433d5b3c9d4beb; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72140052, jitterRate=0.074970543384552}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:54:49,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:49,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb., pid=47, masterSystemTime=1678474489532 2023-03-10 18:54:49,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:49,557 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-10 18:54:49,558 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-10 18:54:49,559 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1912): b7c0739076470c4809433d5b3c9d4beb/family1 is initiating minor compaction (all files) 2023-03-10 18:54:49,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:49,559 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b7c0739076470c4809433d5b3c9d4beb/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:49,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:49,559 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/521abc800cc04ca1943b7ae4cea524a9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/3907738bdaff4aefac572ff9e29f17c5, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2a65cdbd04184d098eb9b35891747ea1_SeqId_46_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp, totalSize=390.1 K 2023-03-10 18:54:49,559 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b7c0739076470c4809433d5b3c9d4beb, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:49,559 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 521abc800cc04ca1943b7ae4cea524a9, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839360 2023-03-10 18:54:49,559 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678474489559"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474489559"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474489559"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474489559"}]},"ts":"1678474489559"} 2023-03-10 18:54:49,560 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 3907738bdaff4aefac572ff9e29f17c5, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239360 2023-03-10 18:54:49,560 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 2a65cdbd04184d098eb9b35891747ea1_SeqId_46_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878839360 2023-03-10 18:54:49,562 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-10 18:54:49,562 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure b7c0739076470c4809433d5b3c9d4beb, server=jenkins-hbase4.apache.org,43181,1678474414959 in 181 msec 2023-03-10 18:54:49,564 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-10 18:54:49,564 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, REOPEN/MOVE in 500 msec 2023-03-10 18:54:49,566 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-10 18:54:49,566 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 505 msec 2023-03-10 18:54:49,568 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 921 msec 2023-03-10 18:54:49,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-10 18:54:49,571 INFO [Listener at localhost/36413] compactions.TestMobCompactor(278): alter status finished 2023-03-10 18:54:49,572 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:49,574 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-10 18:54:49,574 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-10 18:54:49,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:49,575 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475089575 2023-03-10 18:54:49,576 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): b7c0739076470c4809433d5b3c9d4beb#family1#compaction#61 average throughput is 20.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 84.62 MB/second 2023-03-10 18:54:49,576 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:54:49,577 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-10 18:54:49,577 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:49,577 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x493ced45 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:54:49,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b2d83b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:54:49,592 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:49,592 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:49,592 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:49,592 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:49,593 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:50,417 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b1c6f5c2239a4aa3af542d689bc12972 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128b1c6f5c2239a4aa3af542d689bc12972 2023-03-10 18:54:50,417 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:50,430 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:54:50,431 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58968, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:54:50,438 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:54:50,439 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33424, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:54:50,448 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3a0072c117ab4b6c97523cc020a36651 first=Optional[r05] last=Optional[r8] 2023-03-10 18:54:50,448 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=50 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3a0072c117ab4b6c97523cc020a36651}] 2023-03-10 18:54:50,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3a0072c117ab4b6c97523cc020a36651 for inclusion in b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:50,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-10 18:54:50,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:54:50,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:50,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3a0072c117ab4b6c97523cc020a36651 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1fnms9ik5qimj6c6phqhq38rspgoirf7dueh9dp2ms2ofi36o8bohk5qbm2nq679/family1/3a0072c117ab4b6c97523cc020a36651 2023-03-10 18:54:50,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1fnms9ik5qimj6c6phqhq38rspgoirf7dueh9dp2ms2ofi36o8bohk5qbm2nq679/family1/3a0072c117ab4b6c97523cc020a36651 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/5249838f7ccb4b2cb5414d80c07176b8_SeqId_52_ 2023-03-10 18:54:50,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1fnms9ik5qimj6c6phqhq38rspgoirf7dueh9dp2ms2ofi36o8bohk5qbm2nq679/family1/3a0072c117ab4b6c97523cc020a36651 into b7c0739076470c4809433d5b3c9d4beb/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/5249838f7ccb4b2cb5414d80c07176b8_SeqId_52_ - updating store file list. 2023-03-10 18:54:50,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/5249838f7ccb4b2cb5414d80c07176b8_SeqId_52_ into b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:50,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1fnms9ik5qimj6c6phqhq38rspgoirf7dueh9dp2ms2ofi36o8bohk5qbm2nq679/family1/3a0072c117ab4b6c97523cc020a36651 into b7c0739076470c4809433d5b3c9d4beb/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/5249838f7ccb4b2cb5414d80c07176b8_SeqId_52_) 2023-03-10 18:54:50,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1fnms9ik5qimj6c6phqhq38rspgoirf7dueh9dp2ms2ofi36o8bohk5qbm2nq679/family1/3a0072c117ab4b6c97523cc020a36651 2023-03-10 18:54:50,464 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:50,465 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:50,467 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120de8e7a3004d440d5a4aae76fe22ccee9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120de8e7a3004d440d5a4aae76fe22ccee9 2023-03-10 18:54:50,468 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511280b5334e0694546c2acc83330dc6647e9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511280b5334e0694546c2acc83330dc6647e9 2023-03-10 18:54:50,468 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-10 18:54:50,468 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-10 18:54:50,468 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:50,468 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:50,468 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:50,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:50,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:50,470 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-10 18:54:50,471 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 897 msec 2023-03-10 18:54:50,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-10 18:54:50,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:50,538 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-10 18:54:50,539 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-10 18:54:50,539 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-10 18:54:50,541 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-10 18:54:50,543 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, REOPEN/MOVE}] 2023-03-10 18:54:50,544 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, REOPEN/MOVE 2023-03-10 18:54:50,544 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=b7c0739076470c4809433d5b3c9d4beb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:50,545 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678474490544"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474490544"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474490544"}]},"ts":"1678474490544"} 2023-03-10 18:54:50,546 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure b7c0739076470c4809433d5b3c9d4beb, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:54:50,698 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:50,698 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b7c0739076470c4809433d5b3c9d4beb, disabling compactions & flushes 2023-03-10 18:54:50,698 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:50,798 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/.tmp/family1/5d404c5b7d9f4963a413c9546e7741be as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/5d404c5b7d9f4963a413c9546e7741be 2023-03-10 18:54:50,803 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in b7c0739076470c4809433d5b3c9d4beb/family1 of b7c0739076470c4809433d5b3c9d4beb into 5d404c5b7d9f4963a413c9546e7741be(size=193.0 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:54:50,803 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:50,803 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:50,803 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb., storeName=b7c0739076470c4809433d5b3c9d4beb/family1, priority=13, startTime=1678474489556; duration=1sec 2023-03-10 18:54:50,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:50,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. after waiting 0 ms 2023-03-10 18:54:50,803 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:54:50,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:50,806 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/521abc800cc04ca1943b7ae4cea524a9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/3907738bdaff4aefac572ff9e29f17c5, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2a65cdbd04184d098eb9b35891747ea1_SeqId_46_] to archive 2023-03-10 18:54:50,807 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:54:50,808 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/521abc800cc04ca1943b7ae4cea524a9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/521abc800cc04ca1943b7ae4cea524a9 2023-03-10 18:54:50,809 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/3907738bdaff4aefac572ff9e29f17c5 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/3907738bdaff4aefac572ff9e29f17c5 2023-03-10 18:54:50,810 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2a65cdbd04184d098eb9b35891747ea1_SeqId_46_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/2a65cdbd04184d098eb9b35891747ea1_SeqId_46_ 2023-03-10 18:54:50,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-03-10 18:54:50,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:50,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:50,817 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: b7c0739076470c4809433d5b3c9d4beb to self. 2023-03-10 18:54:50,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:50,818 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=b7c0739076470c4809433d5b3c9d4beb, regionState=CLOSED 2023-03-10 18:54:50,818 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678474490818"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474490818"}]},"ts":"1678474490818"} 2023-03-10 18:54:50,821 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-10 18:54:50,821 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure b7c0739076470c4809433d5b3c9d4beb, server=jenkins-hbase4.apache.org,43181,1678474414959 in 273 msec 2023-03-10 18:54:50,821 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=true 2023-03-10 18:54:50,972 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=b7c0739076470c4809433d5b3c9d4beb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:50,972 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678474490971"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474490971"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474490971"}]},"ts":"1678474490971"} 2023-03-10 18:54:50,974 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure b7c0739076470c4809433d5b3c9d4beb, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:54:51,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:51,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b7c0739076470c4809433d5b3c9d4beb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:54:51,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:51,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:51,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:51,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:51,130 INFO [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:51,131 DEBUG [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:51,131 DEBUG [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1 2023-03-10 18:54:51,131 INFO [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b7c0739076470c4809433d5b3c9d4beb columnFamilyName family1 2023-03-10 18:54:51,137 DEBUG [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/5249838f7ccb4b2cb5414d80c07176b8_SeqId_52_ 2023-03-10 18:54:51,139 DEBUG [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/family1/5d404c5b7d9f4963a413c9546e7741be 2023-03-10 18:54:51,140 INFO [StoreOpener-b7c0739076470c4809433d5b3c9d4beb-1] regionserver.HStore(310): Store=b7c0739076470c4809433d5b3c9d4beb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:51,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:51,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:51,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:54:51,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b7c0739076470c4809433d5b3c9d4beb; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61949295, jitterRate=-0.07688356935977936}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:54:51,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:54:51,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb., pid=53, masterSystemTime=1678474491125 2023-03-10 18:54:51,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:51,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:54:51,148 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=b7c0739076470c4809433d5b3c9d4beb, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:51,148 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678474491148"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474491148"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474491148"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474491148"}]},"ts":"1678474491148"} 2023-03-10 18:54:51,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-10 18:54:51,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure b7c0739076470c4809433d5b3c9d4beb, server=jenkins-hbase4.apache.org,43181,1678474414959 in 175 msec 2023-03-10 18:54:51,151 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-10 18:54:51,151 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=b7c0739076470c4809433d5b3c9d4beb, REOPEN/MOVE in 607 msec 2023-03-10 18:54:51,153 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-10 18:54:51,153 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 611 msec 2023-03-10 18:54:51,155 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 627 msec 2023-03-10 18:54:51,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-10 18:54:51,157 INFO [Listener at localhost/36413] compactions.TestMobCompactor(278): alter status finished 2023-03-10 18:54:51,157 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-10 18:54:51,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-10 18:54:51,159 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-10 18:54:51,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:51,160 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475091160 2023-03-10 18:54:51,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:54:51,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-10 18:54:51,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:51,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x57d580ba to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:54:51,166 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47b6e97b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:54:51,166 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:51,166 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:51,166 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:51,166 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:51,166 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:51,166 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:51,166 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:51,167 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-10 18:54:51,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-10 18:54:51,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:54:51,168 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-10 18:54:51,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 10 msec 2023-03-10 18:54:51,226 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=242 (was 258), OpenFileDescriptor=606 (was 634), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=225 (was 260), ProcessCount=170 (was 170), AvailableMemoryMB=5590 (was 5638) 2023-03-10 18:54:51,238 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=242, OpenFileDescriptor=606, MaxFileDescriptor=60000, SystemLoadAverage=225, ProcessCount=170, AvailableMemoryMB=5589 2023-03-10 18:54:51,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-10 18:54:51,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:51,242 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:54:51,243 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-10 18:54:51,243 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:54:51,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-10 18:54:51,244 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:54:51,245 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641 empty. 2023-03-10 18:54:51,245 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:54:51,245 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-10 18:54:51,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-10 18:54:51,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-10 18:54:51,655 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-10 18:54:51,656 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => 75251ceb003f12a816b02e1f9c7cd641, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:54:51,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-10 18:54:52,062 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:52,062 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing 75251ceb003f12a816b02e1f9c7cd641, disabling compactions & flushes 2023-03-10 18:54:52,062 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:52,062 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:52,062 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. after waiting 0 ms 2023-03-10 18:54:52,062 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:52,062 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:52,062 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:54:52,064 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:54:52,065 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678474492065"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474492065"}]},"ts":"1678474492065"} 2023-03-10 18:54:52,067 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-10 18:54:52,067 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:54:52,068 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474492068"}]},"ts":"1678474492068"} 2023-03-10 18:54:52,069 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-10 18:54:52,072 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, ASSIGN}] 2023-03-10 18:54:52,073 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, ASSIGN 2023-03-10 18:54:52,074 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:54:52,225 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=75251ceb003f12a816b02e1f9c7cd641, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:52,225 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678474492225"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474492225"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474492225"}]},"ts":"1678474492225"} 2023-03-10 18:54:52,227 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 75251ceb003f12a816b02e1f9c7cd641, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:54:52,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-10 18:54:52,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:52,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 75251ceb003f12a816b02e1f9c7cd641, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:54:52,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:54:52,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:54:52,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:54:52,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:54:52,383 INFO [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:54:52,384 DEBUG [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:54:52,384 DEBUG [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:54:52,384 INFO [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75251ceb003f12a816b02e1f9c7cd641 columnFamilyName family1 2023-03-10 18:54:52,385 INFO [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] regionserver.HStore(310): Store=75251ceb003f12a816b02e1f9c7cd641/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:54:52,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:54:52,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:54:52,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:54:52,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:54:52,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 75251ceb003f12a816b02e1f9c7cd641; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69479081, jitterRate=0.03531898558139801}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:54:52,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:54:52,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641., pid=57, masterSystemTime=1678474492378 2023-03-10 18:54:52,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:52,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:52,393 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=75251ceb003f12a816b02e1f9c7cd641, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:52,393 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678474492393"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474492393"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474492393"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474492393"}]},"ts":"1678474492393"} 2023-03-10 18:54:52,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-10 18:54:52,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 75251ceb003f12a816b02e1f9c7cd641, server=jenkins-hbase4.apache.org,43181,1678474414959 in 168 msec 2023-03-10 18:54:52,397 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-10 18:54:52,397 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, ASSIGN in 324 msec 2023-03-10 18:54:52,398 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:54:52,398 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474492398"}]},"ts":"1678474492398"} 2023-03-10 18:54:52,399 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-10 18:54:52,402 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:54:52,403 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1620 sec 2023-03-10 18:54:53,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-10 18:54:53,349 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-10 18:54:53,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:53,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:53,376 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:53,376 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:53,376 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-10 18:54:53,376 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-10 18:54:53,376 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:53,376 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:53,378 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:53,378 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:53,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:53,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:53,378 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:53,378 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:53,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:53,379 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:53,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:53,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:53,379 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:53,379 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:53,380 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-10 18:54:53,380 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:53,380 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-10 18:54:53,383 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:53,383 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:53,383 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:53,383 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. started... 2023-03-10 18:54:53,384 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75251ceb003f12a816b02e1f9c7cd641 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-10 18:54:53,802 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130c9a08d07029746569c9178407643cd19 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130c9a08d07029746569c9178407643cd19 2023-03-10 18:54:54,207 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/ba28c14102744a499fdd7c9f9a7da1f5 2023-03-10 18:54:54,212 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/ba28c14102744a499fdd7c9f9a7da1f5 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/ba28c14102744a499fdd7c9f9a7da1f5 2023-03-10 18:54:54,215 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/ba28c14102744a499fdd7c9f9a7da1f5, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-10 18:54:54,216 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 75251ceb003f12a816b02e1f9c7cd641 in 832ms, sequenceid=5, compaction requested=false 2023-03-10 18:54:54,216 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:54,217 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:54:54,217 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:54,217 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:54,217 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:54,217 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,217 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-10 18:54:54,217 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-10 18:54:54,219 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,219 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:54,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:54,219 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,219 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:54,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:54,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:54,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:54,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-10 18:54:54,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24d6c2bc[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:54,220 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-10 18:54:54,220 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,222 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,222 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-10 18:54:54,222 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-10 18:54:54,222 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:54,222 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,222 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:54,224 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-10 18:54:54,224 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,224 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:54,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,224 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-10 18:54:54,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:54,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:54,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:54,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:54,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:54,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:54,227 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-10 18:54:54,232 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-10 18:54:54,232 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:54,232 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:54,232 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,232 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:54,234 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,234 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,234 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:54,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:54,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:54,234 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:54,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:54,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:54,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:54,235 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:54,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,242 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,242 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,242 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,242 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,242 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:54,242 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:54,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:54,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:54,242 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:54,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:54,242 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:54,242 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-10 18:54:54,242 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,242 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,242 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:54,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:54,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:54,243 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:54,343 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:54,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:54,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:54,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:54,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,350 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:54,350 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:54,350 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-10 18:54:54,350 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-10 18:54:54,350 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,350 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,353 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:54,353 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:54,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:54,353 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:54,353 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:54,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,354 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,354 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,354 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-10 18:54:54,354 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:54,354 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-10 18:54:54,354 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:54,354 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-10 18:54:54,354 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:54,355 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:54,355 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:54,355 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. started... 2023-03-10 18:54:54,355 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75251ceb003f12a816b02e1f9c7cd641 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-10 18:54:54,752 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:54:54,770 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ca7852135bda4dc3ae87573e350003cd to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128ca7852135bda4dc3ae87573e350003cd 2023-03-10 18:54:55,176 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/c46aedbf5b68409a8caa6f545f7ad733 2023-03-10 18:54:55,180 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/c46aedbf5b68409a8caa6f545f7ad733 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c46aedbf5b68409a8caa6f545f7ad733 2023-03-10 18:54:55,184 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c46aedbf5b68409a8caa6f545f7ad733, entries=1, sequenceid=9, filesize=5.0 K 2023-03-10 18:54:55,185 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 75251ceb003f12a816b02e1f9c7cd641 in 830ms, sequenceid=9, compaction requested=false 2023-03-10 18:54:55,185 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:54:55,185 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:55,185 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:55,185 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:55,185 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,185 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-10 18:54:55,185 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-10 18:54:55,187 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,187 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:55,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:55,187 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,187 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:55,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:55,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:55,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:55,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-10 18:54:55,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56ae23ee[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:55,188 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-10 18:54:55,188 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,190 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,190 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-10 18:54:55,190 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-10 18:54:55,190 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,190 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:55,190 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:55,191 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-10 18:54:55,191 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,191 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:55,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,191 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-10 18:54:55,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:55,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:55,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:55,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:55,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:55,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:55,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-10 18:54:55,193 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-10 18:54:55,193 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:55,193 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:55,193 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,193 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:55,195 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,195 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:55,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:55,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,196 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:55,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:55,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:55,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:55,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,196 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:55,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,197 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:55,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,199 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,199 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,199 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:55,199 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:55,199 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:55,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:55,199 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:55,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:55,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:55,199 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,199 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,199 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:55,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:55,199 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,200 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:55,200 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:55,300 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:55,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:55,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:55,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:55,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,306 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:55,306 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:55,306 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-10 18:54:55,306 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-10 18:54:55,306 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,306 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,308 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,308 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:55,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:55,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:55,308 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:55,308 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:55,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:55,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-10 18:54:55,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:55,309 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-10 18:54:55,309 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:55,309 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-10 18:54:55,309 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:55,309 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:55,309 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:55,309 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. started... 2023-03-10 18:54:55,309 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75251ceb003f12a816b02e1f9c7cd641 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:55,722 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201c4e9424d12824b0783ade6be6e383d7f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c4e9424d12824b0783ade6be6e383d7f 2023-03-10 18:54:56,128 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/0bf03c7417914915b3ea527a2e881f37 2023-03-10 18:54:56,132 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/0bf03c7417914915b3ea527a2e881f37 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0bf03c7417914915b3ea527a2e881f37 2023-03-10 18:54:56,138 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0bf03c7417914915b3ea527a2e881f37, entries=1, sequenceid=13, filesize=5.0 K 2023-03-10 18:54:56,138 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75251ceb003f12a816b02e1f9c7cd641 in 829ms, sequenceid=13, compaction requested=true 2023-03-10 18:54:56,139 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:54:56,139 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:56,139 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:56,139 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:56,139 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,139 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-10 18:54:56,139 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-10 18:54:56,140 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:56,141 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:56,141 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,141 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:56,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-10 18:54:56,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44f02e8b[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:56,142 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-10 18:54:56,142 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,144 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,144 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-10 18:54:56,144 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-10 18:54:56,144 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:56,144 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,144 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:56,146 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-10 18:54:56,146 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,146 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:56,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:56,146 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-10 18:54:56,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:56,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:56,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:56,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-10 18:54:56,148 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-10 18:54:56,148 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:56,148 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:56,148 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,148 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:56,150 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,150 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:56,150 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:56,150 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,151 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:56,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,154 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:56,154 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:56,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:56,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:56,154 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:56,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-10 18:54:56,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:56,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:56,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:56,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:56,154 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,154 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,154 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,154 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,154 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,154 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,154 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,154 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:56,154 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:56,254 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:56,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:56,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:56,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:56,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,261 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:56,261 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:56,261 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-10 18:54:56,261 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-10 18:54:56,261 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,261 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,263 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:56,263 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:56,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:56,264 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:56,264 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:56,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,264 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:56,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,265 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,265 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-10 18:54:56,265 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:56,265 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-10 18:54:56,265 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:56,265 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-10 18:54:56,265 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:56,265 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:56,265 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:56,265 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. started... 2023-03-10 18:54:56,266 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75251ceb003f12a816b02e1f9c7cd641 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:56,677 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205aab5d5fceec545bc9a57fe4046eca9d4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205aab5d5fceec545bc9a57fe4046eca9d4 2023-03-10 18:54:57,083 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/0d6fb4eb854646798dc2b660deb010a9 2023-03-10 18:54:57,088 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/0d6fb4eb854646798dc2b660deb010a9 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0d6fb4eb854646798dc2b660deb010a9 2023-03-10 18:54:57,091 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0d6fb4eb854646798dc2b660deb010a9, entries=1, sequenceid=17, filesize=5.0 K 2023-03-10 18:54:57,092 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75251ceb003f12a816b02e1f9c7cd641 in 826ms, sequenceid=17, compaction requested=true 2023-03-10 18:54:57,092 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:54:57,092 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:57,092 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:57,092 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:57,092 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,092 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-10 18:54:57,092 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-10 18:54:57,094 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,094 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:57,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:57,094 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,094 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:57,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:57,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:57,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:57,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-10 18:54:57,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a8fb626[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:57,096 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-10 18:54:57,096 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,100 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,100 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-10 18:54:57,100 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-10 18:54:57,100 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:57,100 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,100 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:57,102 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,102 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-10 18:54:57,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:57,102 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:57,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:57,102 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-10 18:54:57,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:57,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:57,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:57,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:57,105 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-10 18:54:57,105 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-10 18:54:57,105 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:57,105 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:57,105 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,105 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:57,107 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,107 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:57,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:57,107 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:57,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:57,107 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:57,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:57,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:57,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,109 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:57,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,111 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,111 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,111 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:57,111 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:57,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:57,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:57,111 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:57,111 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-10 18:54:57,111 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:57,112 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:57,112 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:57,112 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:57,112 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,112 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:57,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:57,212 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:57,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:57,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:57,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:57,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,218 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:57,218 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:57,218 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-10 18:54:57,218 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-10 18:54:57,219 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,219 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,220 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,220 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:57,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:57,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:57,221 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:57,221 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:57,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:57,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,221 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,222 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-10 18:54:57,222 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-10 18:54:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-10 18:54:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:57,222 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:57,222 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. started... 2023-03-10 18:54:57,222 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75251ceb003f12a816b02e1f9c7cd641 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:57,634 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122800eb05dfd0d1403688b99b62baf016a5 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122800eb05dfd0d1403688b99b62baf016a5 2023-03-10 18:54:58,040 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/4240861b879841c0b26b7e9bb20b1433 2023-03-10 18:54:58,044 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/4240861b879841c0b26b7e9bb20b1433 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/4240861b879841c0b26b7e9bb20b1433 2023-03-10 18:54:58,048 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/4240861b879841c0b26b7e9bb20b1433, entries=1, sequenceid=21, filesize=5.0 K 2023-03-10 18:54:58,049 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75251ceb003f12a816b02e1f9c7cd641 in 827ms, sequenceid=21, compaction requested=true 2023-03-10 18:54:58,049 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:54:58,049 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:58,049 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:58,049 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:58,049 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,049 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-10 18:54:58,049 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-10 18:54:58,051 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,051 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:58,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:58,052 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,052 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:58,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:58,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:58,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:58,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-10 18:54:58,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18de56e9[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:58,053 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-10 18:54:58,053 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,054 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,054 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,054 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-10 18:54:58,055 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-10 18:54:58,054 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:58,055 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:58,056 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,056 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-10 18:54:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:58,056 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:58,056 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-10 18:54:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:58,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-10 18:54:58,058 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-10 18:54:58,058 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:58,058 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:58,058 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,058 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:58,059 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:58,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:58,059 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,059 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:58,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:58,060 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:58,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:58,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:58,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,061 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:58,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:58,064 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:58,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:58,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:58,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:58,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:58,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:58,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:58,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,064 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:58,064 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,064 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:58,165 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:58,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:58,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:58,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:58,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,171 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:58,171 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:58,171 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-10 18:54:58,171 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-10 18:54:58,171 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,171 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,173 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:58,173 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:58,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:58,173 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:58,173 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:58,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,174 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-10 18:54:58,174 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:58,174 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-10 18:54:58,174 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:58,174 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-10 18:54:58,174 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:58,174 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:58,174 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:58,174 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. started... 2023-03-10 18:54:58,174 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75251ceb003f12a816b02e1f9c7cd641 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:58,587 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512315944f52105444e7889476eeabbc19973 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512315944f52105444e7889476eeabbc19973 2023-03-10 18:54:58,992 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/9e252f98a2a9448e8f00f24f8ecf487b 2023-03-10 18:54:58,997 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/9e252f98a2a9448e8f00f24f8ecf487b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/9e252f98a2a9448e8f00f24f8ecf487b 2023-03-10 18:54:59,002 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/9e252f98a2a9448e8f00f24f8ecf487b, entries=1, sequenceid=25, filesize=5.0 K 2023-03-10 18:54:59,002 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75251ceb003f12a816b02e1f9c7cd641 in 828ms, sequenceid=25, compaction requested=true 2023-03-10 18:54:59,003 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:54:59,003 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:59,003 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:59,003 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:59,003 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,003 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-10 18:54:59,003 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-10 18:54:59,005 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,005 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:59,005 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,005 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:59,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-10 18:54:59,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f48a74f[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:59,006 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-10 18:54:59,006 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,007 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,008 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-10 18:54:59,008 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-10 18:54:59,008 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:59,008 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,008 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:59,010 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-10 18:54:59,010 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,010 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:59,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,010 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-10 18:54:59,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:59,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:59,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:59,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:59,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:59,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:59,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-10 18:54:59,012 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-10 18:54:59,012 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:59,012 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:59,012 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,012 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:59,014 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:59,014 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,014 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:59,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:59,015 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,016 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,021 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,021 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:59,021 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:59,021 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:59,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:59,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:59,021 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:59,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:59,021 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,021 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,021 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,021 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,021 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,022 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:59,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:59,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:59,022 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:59,122 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:54:59,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:54:59,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:54:59,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-10 18:54:59,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,128 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:59,128 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:59,128 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-10 18:54:59,128 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-10 18:54:59,129 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,129 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,134 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,134 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:59,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:59,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:59,134 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,134 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:54:59,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,135 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:54:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,135 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,135 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-10 18:54:59,135 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,135 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-10 18:54:59,135 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:54:59,135 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-10 18:54:59,135 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:54:59,135 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:54:59,136 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:59,136 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. started... 2023-03-10 18:54:59,136 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75251ceb003f12a816b02e1f9c7cd641 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:54:59,548 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010192c82ebc093849d2acf3c92e40c04153 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010192c82ebc093849d2acf3c92e40c04153 2023-03-10 18:54:59,954 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/52dfbd133cdb47818fef5ee2425d15eb 2023-03-10 18:54:59,958 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/52dfbd133cdb47818fef5ee2425d15eb as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/52dfbd133cdb47818fef5ee2425d15eb 2023-03-10 18:54:59,962 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/52dfbd133cdb47818fef5ee2425d15eb, entries=1, sequenceid=29, filesize=5.0 K 2023-03-10 18:54:59,962 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75251ceb003f12a816b02e1f9c7cd641 in 826ms, sequenceid=29, compaction requested=true 2023-03-10 18:54:59,963 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:54:59,963 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:54:59,963 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:54:59,963 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:54:59,963 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,963 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-10 18:54:59,963 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-10 18:54:59,964 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,964 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:59,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:59,965 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,965 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:54:59,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:59,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:59,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:59,966 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-10 18:54:59,966 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a89425b[Count = 0] remaining members to acquire global barrier 2023-03-10 18:54:59,966 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-10 18:54:59,966 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,973 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,973 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-10 18:54:59,973 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-10 18:54:59,973 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:54:59,974 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,974 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:54:59,975 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,975 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-10 18:54:59,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:59,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:59,975 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:59,975 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-10 18:54:59,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:59,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:59,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:59,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:54:59,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-10 18:54:59,977 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-10 18:54:59,977 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:54:59,977 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:54:59,977 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,977 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:54:59,979 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,979 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:54:59,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:54:59,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,979 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:59,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:54:59,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:59,980 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:54:59,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,981 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:54:59,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,983 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:54:59,983 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:54:59,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:54:59,983 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:54:59,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:54:59,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:54:59,983 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,983 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:54:59,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:54:59,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:54:59,983 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,983 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:54:59,983 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-10 18:54:59,983 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:54:59,983 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,984 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:54:59,984 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,084 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:00,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:00,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:00,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-10 18:55:00,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,090 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:55:00,090 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:00,090 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-10 18:55:00,090 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-10 18:55:00,090 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,090 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,092 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,092 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:00,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:00,093 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,093 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,093 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,094 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-10 18:55:00,094 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,094 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-10 18:55:00,094 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:00,094 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-10 18:55:00,094 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:55:00,094 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:00,094 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:00,094 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. started... 2023-03-10 18:55:00,094 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75251ceb003f12a816b02e1f9c7cd641 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:55:00,507 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030eca4e7b2e65463c8794fbd726d162c1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601030eca4e7b2e65463c8794fbd726d162c1 2023-03-10 18:55:00,912 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/d22b3b2e3fbd4907af690eb42161d18e 2023-03-10 18:55:00,917 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/d22b3b2e3fbd4907af690eb42161d18e as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/d22b3b2e3fbd4907af690eb42161d18e 2023-03-10 18:55:00,921 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/d22b3b2e3fbd4907af690eb42161d18e, entries=1, sequenceid=33, filesize=5.0 K 2023-03-10 18:55:00,922 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75251ceb003f12a816b02e1f9c7cd641 in 828ms, sequenceid=33, compaction requested=true 2023-03-10 18:55:00,923 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:00,923 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:00,923 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:55:00,923 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:55:00,923 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,923 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-10 18:55:00,923 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-10 18:55:00,925 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,925 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:00,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:00,925 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,925 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:00,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:00,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:00,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-10 18:55:00,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@dc0e5aa[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:00,926 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-10 18:55:00,927 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,928 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,928 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-10 18:55:00,928 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-10 18:55:00,928 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:00,928 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,928 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:00,929 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,929 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-10 18:55:00,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,929 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:00,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:00,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:00,930 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-10 18:55:00,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:00,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:00,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:00,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:00,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-10 18:55:00,932 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-10 18:55:00,932 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:00,932 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:00,932 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,932 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:00,934 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,934 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:00,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,935 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:00,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:00,935 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:00,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,936 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:00,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,939 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,939 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:00,939 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:00,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:00,939 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,939 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:00,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:55:00,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:00,939 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:00,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:00,939 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:00,939 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,939 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:00,939 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:00,939 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:00,939 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:01,040 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:01,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:01,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:01,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-10 18:55:01,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,047 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:55:01,047 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:01,047 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-10 18:55:01,047 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-10 18:55:01,047 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,047 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,049 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,049 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:01,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:01,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:01,049 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,049 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:01,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:01,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,056 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-10 18:55:01,056 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,056 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-10 18:55:01,056 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:01,056 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-10 18:55:01,056 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:55:01,056 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:01,056 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:01,056 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. started... 2023-03-10 18:55:01,057 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75251ceb003f12a816b02e1f9c7cd641 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:55:01,474 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509079da942fbb0ab4fc79693a3320fcaf0c2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201509079da942fbb0ab4fc79693a3320fcaf0c2 2023-03-10 18:55:01,479 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/2017747a2aab470d8bd76fc5303ddb23 2023-03-10 18:55:01,483 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/2017747a2aab470d8bd76fc5303ddb23 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/2017747a2aab470d8bd76fc5303ddb23 2023-03-10 18:55:01,487 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/2017747a2aab470d8bd76fc5303ddb23, entries=1, sequenceid=37, filesize=5.0 K 2023-03-10 18:55:01,487 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75251ceb003f12a816b02e1f9c7cd641 in 430ms, sequenceid=37, compaction requested=true 2023-03-10 18:55:01,488 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:01,488 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:01,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:55:01,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:55:01,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-10 18:55:01,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-10 18:55:01,491 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,491 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:01,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:01,492 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,492 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:01,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:01,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:01,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:01,493 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-10 18:55:01,493 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67c383e5[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:01,493 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-10 18:55:01,493 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,494 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,494 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-10 18:55:01,494 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-10 18:55:01,494 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:01,494 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,494 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:01,496 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-10 18:55:01,496 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:01,496 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,496 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-10 18:55:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:01,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:01,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:01,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-10 18:55:01,498 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-10 18:55:01,498 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:01,498 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:01,498 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,498 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:01,500 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,500 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,500 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:01,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:01,500 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,501 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:01,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:01,504 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:01,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:01,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:01,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,504 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-10 18:55:01,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:01,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:55:01,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:01,505 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:01,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:01,505 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,505 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:01,505 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:01,605 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:01,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:01,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:01,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-10 18:55:01,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,611 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:55:01,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:01,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-10 18:55:01,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-10 18:55:01,612 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,612 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,613 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,613 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:01,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:01,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:01,613 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:01,613 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,614 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,614 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,614 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-10 18:55:01,615 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-10 18:55:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-10 18:55:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:55:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:01,615 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:01,615 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. started... 2023-03-10 18:55:01,615 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75251ceb003f12a816b02e1f9c7cd641 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:55:02,033 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120778ffd4af61f43f4b0c960bd50150920 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120778ffd4af61f43f4b0c960bd50150920 2023-03-10 18:55:02,439 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/c6cb9810e3b748ceaf986c963fff8fab 2023-03-10 18:55:02,444 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/c6cb9810e3b748ceaf986c963fff8fab as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c6cb9810e3b748ceaf986c963fff8fab 2023-03-10 18:55:02,450 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c6cb9810e3b748ceaf986c963fff8fab, entries=1, sequenceid=41, filesize=5.0 K 2023-03-10 18:55:02,451 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75251ceb003f12a816b02e1f9c7cd641 in 836ms, sequenceid=41, compaction requested=true 2023-03-10 18:55:02,451 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:02,451 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:55:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:55:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-10 18:55:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-10 18:55:02,453 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,453 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:02,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:02,454 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,454 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:02,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-10 18:55:02,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d4b328d[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:02,455 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-10 18:55:02,455 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,456 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,456 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-10 18:55:02,457 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-10 18:55:02,457 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,457 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:02,457 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:02,459 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,459 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-10 18:55:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:02,459 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:02,459 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-10 18:55:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:02,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-10 18:55:02,461 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-10 18:55:02,461 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:02,461 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:02,461 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,461 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:02,463 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,463 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,463 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:02,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:02,463 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,465 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:02,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,470 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,470 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:02,470 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-10 18:55:02,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-10 18:55:02,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:02,471 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:02,471 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:02,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:02,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:02,471 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,471 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:02,471 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,471 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,471 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:02,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:02,471 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:02,471 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:02,571 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:02,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:02,574 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:02,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-10 18:55:02,576 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-10 18:55:02,576 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:55:02,576 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475102576 2023-03-10 18:55:02,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:55:02,580 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-10 18:55:02,580 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:02,581 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x0fa6d5ca to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:55:02,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79ecc0d1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:55:02,588 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:03,426 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031f578f615c6d4cc9a2f035878eae27e4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601031f578f615c6d4cc9a2f035878eae27e4 2023-03-10 18:55:03,427 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:03,439 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:55:03,441 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39766, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:55:03,450 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:55:03,451 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44678, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:55:03,460 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6c4007c17fe34ebf87072a438110d248 first=Optional[r3] last=Optional[r6] 2023-03-10 18:55:03,461 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6c4007c17fe34ebf87072a438110d248}] 2023-03-10 18:55:03,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6c4007c17fe34ebf87072a438110d248 for inclusion in 75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:55:03,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-10 18:55:03,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:55:03,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:03,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6c4007c17fe34ebf87072a438110d248 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ctcusk6rvvbmg7p636hjei2r4uoj0sjqgip8v5mg7sfm6k8soaj69vclvv3m0fm1/family1/6c4007c17fe34ebf87072a438110d248 2023-03-10 18:55:03,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ctcusk6rvvbmg7p636hjei2r4uoj0sjqgip8v5mg7sfm6k8soaj69vclvv3m0fm1/family1/6c4007c17fe34ebf87072a438110d248 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/46e8e32204db48d7bb0fd62edd28514d_SeqId_44_ 2023-03-10 18:55:03,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ctcusk6rvvbmg7p636hjei2r4uoj0sjqgip8v5mg7sfm6k8soaj69vclvv3m0fm1/family1/6c4007c17fe34ebf87072a438110d248 into 75251ceb003f12a816b02e1f9c7cd641/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/46e8e32204db48d7bb0fd62edd28514d_SeqId_44_ - updating store file list. 2023-03-10 18:55:03,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/46e8e32204db48d7bb0fd62edd28514d_SeqId_44_ into 75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:55:03,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ctcusk6rvvbmg7p636hjei2r4uoj0sjqgip8v5mg7sfm6k8soaj69vclvv3m0fm1/family1/6c4007c17fe34ebf87072a438110d248 into 75251ceb003f12a816b02e1f9c7cd641/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/46e8e32204db48d7bb0fd62edd28514d_SeqId_44_) 2023-03-10 18:55:03,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ctcusk6rvvbmg7p636hjei2r4uoj0sjqgip8v5mg7sfm6k8soaj69vclvv3m0fm1/family1/6c4007c17fe34ebf87072a438110d248 2023-03-10 18:55:03,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-10 18:55:03,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-10 18:55:03,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): 75251ceb003f12a816b02e1f9c7cd641/family1 is initiating minor compaction 2023-03-10 18:55:03,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6fbd4a27; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:55:03,480 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 75251ceb003f12a816b02e1f9c7cd641/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:03,480 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c46aedbf5b68409a8caa6f545f7ad733, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0bf03c7417914915b3ea527a2e881f37, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0d6fb4eb854646798dc2b660deb010a9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/4240861b879841c0b26b7e9bb20b1433, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/9e252f98a2a9448e8f00f24f8ecf487b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/52dfbd133cdb47818fef5ee2425d15eb, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/d22b3b2e3fbd4907af690eb42161d18e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/2017747a2aab470d8bd76fc5303ddb23, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c6cb9810e3b748ceaf986c963fff8fab, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/46e8e32204db48d7bb0fd62edd28514d_SeqId_44_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp, totalSize=50.3 K 2023-03-10 18:55:03,480 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting c46aedbf5b68409a8caa6f545f7ad733, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-10 18:55:03,480 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 0bf03c7417914915b3ea527a2e881f37, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-10 18:55:03,480 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 0d6fb4eb854646798dc2b660deb010a9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-10 18:55:03,480 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 4240861b879841c0b26b7e9bb20b1433, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-10 18:55:03,480 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 9e252f98a2a9448e8f00f24f8ecf487b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-10 18:55:03,480 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 52dfbd133cdb47818fef5ee2425d15eb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-10 18:55:03,480 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting d22b3b2e3fbd4907af690eb42161d18e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-10 18:55:03,480 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 2017747a2aab470d8bd76fc5303ddb23, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-10 18:55:03,480 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting c6cb9810e3b748ceaf986c963fff8fab, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-10 18:55:03,481 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 46e8e32204db48d7bb0fd62edd28514d_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-10 18:55:03,482 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:03,483 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:03,485 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122800eb05dfd0d1403688b99b62baf016a5 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122800eb05dfd0d1403688b99b62baf016a5 2023-03-10 18:55:03,486 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512315944f52105444e7889476eeabbc19973 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512315944f52105444e7889476eeabbc19973 2023-03-10 18:55:03,488 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010192c82ebc093849d2acf3c92e40c04153 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010192c82ebc093849d2acf3c92e40c04153 2023-03-10 18:55:03,490 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601030eca4e7b2e65463c8794fbd726d162c1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601030eca4e7b2e65463c8794fbd726d162c1 2023-03-10 18:55:03,490 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-10 18:55:03,490 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-10 18:55:03,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:03,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:03,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:03,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:03,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:03,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:03,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:03,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:03,520 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): 75251ceb003f12a816b02e1f9c7cd641#family1#compaction#72 average throughput is 0.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 84.62 MB/second 2023-03-10 18:55:04,340 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/09f66ec506484a19b27d087307d06343 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/09f66ec506484a19b27d087307d06343 2023-03-10 18:55:04,344 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 75251ceb003f12a816b02e1f9c7cd641/family1 of 75251ceb003f12a816b02e1f9c7cd641 into 09f66ec506484a19b27d087307d06343(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:55:04,344 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:04,344 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678474503480; duration=0sec 2023-03-10 18:55:04,344 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:55:04,352 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ee7bd7768d39402f8517a3a75927d3f8 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205ee7bd7768d39402f8517a3a75927d3f8 2023-03-10 18:55:04,353 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:04,375 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9384222304764ff5a37f6261cb749449 first=Optional[r00] last=Optional[r2] 2023-03-10 18:55:04,376 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9384222304764ff5a37f6261cb749449}] 2023-03-10 18:55:04,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9384222304764ff5a37f6261cb749449 for inclusion in 75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:55:04,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-10 18:55:04,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:55:04,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:04,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9384222304764ff5a37f6261cb749449 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__spg12llsrofd06ej2uphvf988mlkcg8lggje6lkjoeagv3jvet27fffgdaoo3iiq/family1/9384222304764ff5a37f6261cb749449 2023-03-10 18:55:04,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__spg12llsrofd06ej2uphvf988mlkcg8lggje6lkjoeagv3jvet27fffgdaoo3iiq/family1/9384222304764ff5a37f6261cb749449 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/89a03bb0eba841c88bff78e2415ef6a0_SeqId_47_ 2023-03-10 18:55:04,384 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__spg12llsrofd06ej2uphvf988mlkcg8lggje6lkjoeagv3jvet27fffgdaoo3iiq/family1/9384222304764ff5a37f6261cb749449 into 75251ceb003f12a816b02e1f9c7cd641/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/89a03bb0eba841c88bff78e2415ef6a0_SeqId_47_ - updating store file list. 2023-03-10 18:55:04,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/89a03bb0eba841c88bff78e2415ef6a0_SeqId_47_ into 75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:55:04,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__spg12llsrofd06ej2uphvf988mlkcg8lggje6lkjoeagv3jvet27fffgdaoo3iiq/family1/9384222304764ff5a37f6261cb749449 into 75251ceb003f12a816b02e1f9c7cd641/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/89a03bb0eba841c88bff78e2415ef6a0_SeqId_47_) 2023-03-10 18:55:04,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__spg12llsrofd06ej2uphvf988mlkcg8lggje6lkjoeagv3jvet27fffgdaoo3iiq/family1/9384222304764ff5a37f6261cb749449 2023-03-10 18:55:04,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-10 18:55:04,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-10 18:55:04,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): 75251ceb003f12a816b02e1f9c7cd641/family1 is initiating minor compaction (all files) 2023-03-10 18:55:04,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3447fb27; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:55:04,389 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 75251ceb003f12a816b02e1f9c7cd641/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:04,389 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/ba28c14102744a499fdd7c9f9a7da1f5, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/09f66ec506484a19b27d087307d06343, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/89a03bb0eba841c88bff78e2415ef6a0_SeqId_47_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp, totalSize=388.1 K 2023-03-10 18:55:04,390 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting ba28c14102744a499fdd7c9f9a7da1f5, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839360 2023-03-10 18:55:04,390 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 09f66ec506484a19b27d087307d06343, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239360 2023-03-10 18:55:04,391 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 89a03bb0eba841c88bff78e2415ef6a0_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878839360 2023-03-10 18:55:04,391 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:04,392 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:04,393 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130c9a08d07029746569c9178407643cd19 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130c9a08d07029746569c9178407643cd19 2023-03-10 18:55:04,394 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c4e9424d12824b0783ade6be6e383d7f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c4e9424d12824b0783ade6be6e383d7f 2023-03-10 18:55:04,395 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205aab5d5fceec545bc9a57fe4046eca9d4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205aab5d5fceec545bc9a57fe4046eca9d4 2023-03-10 18:55:04,395 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-10 18:55:04,395 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-10 18:55:04,396 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:04,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:04,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:55:04,397 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-10 18:55:04,398 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8230 sec 2023-03-10 18:55:04,412 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): 75251ceb003f12a816b02e1f9c7cd641#family1#compaction#73 average throughput is 25.66 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 84.62 MB/second 2023-03-10 18:55:04,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c46aedbf5b68409a8caa6f545f7ad733, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0bf03c7417914915b3ea527a2e881f37, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0d6fb4eb854646798dc2b660deb010a9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/4240861b879841c0b26b7e9bb20b1433, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/9e252f98a2a9448e8f00f24f8ecf487b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/52dfbd133cdb47818fef5ee2425d15eb, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/d22b3b2e3fbd4907af690eb42161d18e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/2017747a2aab470d8bd76fc5303ddb23, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c6cb9810e3b748ceaf986c963fff8fab, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/46e8e32204db48d7bb0fd62edd28514d_SeqId_44_] to archive 2023-03-10 18:55:04,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:04,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c46aedbf5b68409a8caa6f545f7ad733 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c46aedbf5b68409a8caa6f545f7ad733 2023-03-10 18:55:04,443 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0bf03c7417914915b3ea527a2e881f37 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0bf03c7417914915b3ea527a2e881f37 2023-03-10 18:55:04,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0d6fb4eb854646798dc2b660deb010a9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/0d6fb4eb854646798dc2b660deb010a9 2023-03-10 18:55:04,446 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/4240861b879841c0b26b7e9bb20b1433 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/4240861b879841c0b26b7e9bb20b1433 2023-03-10 18:55:04,447 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/9e252f98a2a9448e8f00f24f8ecf487b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/9e252f98a2a9448e8f00f24f8ecf487b 2023-03-10 18:55:04,448 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/52dfbd133cdb47818fef5ee2425d15eb to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/52dfbd133cdb47818fef5ee2425d15eb 2023-03-10 18:55:04,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/d22b3b2e3fbd4907af690eb42161d18e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/d22b3b2e3fbd4907af690eb42161d18e 2023-03-10 18:55:04,451 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/2017747a2aab470d8bd76fc5303ddb23 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/2017747a2aab470d8bd76fc5303ddb23 2023-03-10 18:55:04,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c6cb9810e3b748ceaf986c963fff8fab to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/c6cb9810e3b748ceaf986c963fff8fab 2023-03-10 18:55:04,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/46e8e32204db48d7bb0fd62edd28514d_SeqId_44_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/46e8e32204db48d7bb0fd62edd28514d_SeqId_44_ 2023-03-10 18:55:04,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-10 18:55:04,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:04,492 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-10 18:55:04,493 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-10 18:55:04,493 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-10 18:55:04,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-10 18:55:04,498 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, REOPEN/MOVE}] 2023-03-10 18:55:04,499 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, REOPEN/MOVE 2023-03-10 18:55:04,499 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=75251ceb003f12a816b02e1f9c7cd641, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:04,500 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678474504499"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474504499"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474504499"}]},"ts":"1678474504499"} 2023-03-10 18:55:04,501 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 75251ceb003f12a816b02e1f9c7cd641, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:04,652 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:04,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 75251ceb003f12a816b02e1f9c7cd641, disabling compactions & flushes 2023-03-10 18:55:04,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:05,267 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/.tmp/family1/bf366c4e463749c18fe49a9f2de92afb as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/bf366c4e463749c18fe49a9f2de92afb 2023-03-10 18:55:05,271 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 75251ceb003f12a816b02e1f9c7cd641/family1 of 75251ceb003f12a816b02e1f9c7cd641 into bf366c4e463749c18fe49a9f2de92afb(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:55:05,272 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:05,272 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:05,272 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1678474504389; duration=0sec 2023-03-10 18:55:05,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:05,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. after waiting 0 ms 2023-03-10 18:55:05,272 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:55:05,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:05,274 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/ba28c14102744a499fdd7c9f9a7da1f5, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/09f66ec506484a19b27d087307d06343, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/89a03bb0eba841c88bff78e2415ef6a0_SeqId_47_] to archive 2023-03-10 18:55:05,275 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:05,276 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/ba28c14102744a499fdd7c9f9a7da1f5 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/ba28c14102744a499fdd7c9f9a7da1f5 2023-03-10 18:55:05,277 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/09f66ec506484a19b27d087307d06343 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/09f66ec506484a19b27d087307d06343 2023-03-10 18:55:05,279 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/89a03bb0eba841c88bff78e2415ef6a0_SeqId_47_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/89a03bb0eba841c88bff78e2415ef6a0_SeqId_47_ 2023-03-10 18:55:05,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-10 18:55:05,283 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:05,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:05,283 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 75251ceb003f12a816b02e1f9c7cd641 to self. 2023-03-10 18:55:05,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:05,285 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=75251ceb003f12a816b02e1f9c7cd641, regionState=CLOSED 2023-03-10 18:55:05,285 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678474505285"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474505285"}]},"ts":"1678474505285"} 2023-03-10 18:55:05,287 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-10 18:55:05,287 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 75251ceb003f12a816b02e1f9c7cd641, server=jenkins-hbase4.apache.org,43181,1678474414959 in 785 msec 2023-03-10 18:55:05,287 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=true 2023-03-10 18:55:05,438 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=75251ceb003f12a816b02e1f9c7cd641, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:05,438 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678474505438"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474505438"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474505438"}]},"ts":"1678474505438"} 2023-03-10 18:55:05,440 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 75251ceb003f12a816b02e1f9c7cd641, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:05,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:05,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 75251ceb003f12a816b02e1f9c7cd641, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:55:05,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:05,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:05,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:05,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:05,596 INFO [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:05,597 DEBUG [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:55:05,597 DEBUG [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:55:05,597 INFO [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75251ceb003f12a816b02e1f9c7cd641 columnFamilyName family1 2023-03-10 18:55:05,601 DEBUG [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/bf366c4e463749c18fe49a9f2de92afb 2023-03-10 18:55:05,601 INFO [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] regionserver.HStore(310): Store=75251ceb003f12a816b02e1f9c7cd641/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:05,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:05,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:05,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:05,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 75251ceb003f12a816b02e1f9c7cd641; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58823637, jitterRate=-0.12345950305461884}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:55:05,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:05,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641., pid=63, masterSystemTime=1678474505591 2023-03-10 18:55:05,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:05,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:05,608 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=75251ceb003f12a816b02e1f9c7cd641, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:05,608 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678474505608"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474505608"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474505608"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474505608"}]},"ts":"1678474505608"} 2023-03-10 18:55:05,610 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-10 18:55:05,611 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 75251ceb003f12a816b02e1f9c7cd641, server=jenkins-hbase4.apache.org,43181,1678474414959 in 169 msec 2023-03-10 18:55:05,612 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-10 18:55:05,612 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, REOPEN/MOVE in 1.1120 sec 2023-03-10 18:55:05,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-10 18:55:05,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1170 sec 2023-03-10 18:55:05,615 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1320 sec 2023-03-10 18:55:05,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-10 18:55:05,617 INFO [Listener at localhost/36413] compactions.TestMobCompactor(278): alter status finished 2023-03-10 18:55:05,617 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:05,618 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-10 18:55:05,619 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-10 18:55:05,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:55:05,620 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475105620 2023-03-10 18:55:05,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:55:05,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-10 18:55:05,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:05,621 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x5ed14a21 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:55:05,626 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@31b533a9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:55:05,627 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:05,627 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:05,627 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:05,627 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:05,627 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:06,452 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ed8e47bc5e574dcda6681d1f4235af4b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128ed8e47bc5e574dcda6681d1f4235af4b 2023-03-10 18:55:06,453 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:06,465 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:55:06,467 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39774, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:55:06,472 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:55:06,474 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44692, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:55:06,482 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/19d657ed17a44a9f9ae6517dd9a8d8d9 first=Optional[r05] last=Optional[r8] 2023-03-10 18:55:06,482 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=51 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/19d657ed17a44a9f9ae6517dd9a8d8d9}] 2023-03-10 18:55:06,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/19d657ed17a44a9f9ae6517dd9a8d8d9 for inclusion in 75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:55:06,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-10 18:55:06,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:55:06,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:06,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/19d657ed17a44a9f9ae6517dd9a8d8d9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__v2g2cnd2vl1pu8fu5qqc59bujrke3s4i6r7psag2e3fsef7akp2n10qjptkvb6n5/family1/19d657ed17a44a9f9ae6517dd9a8d8d9 2023-03-10 18:55:06,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__v2g2cnd2vl1pu8fu5qqc59bujrke3s4i6r7psag2e3fsef7akp2n10qjptkvb6n5/family1/19d657ed17a44a9f9ae6517dd9a8d8d9 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/b3e5540101b147b4b22b2a57f8ce6fbf_SeqId_53_ 2023-03-10 18:55:06,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__v2g2cnd2vl1pu8fu5qqc59bujrke3s4i6r7psag2e3fsef7akp2n10qjptkvb6n5/family1/19d657ed17a44a9f9ae6517dd9a8d8d9 into 75251ceb003f12a816b02e1f9c7cd641/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/b3e5540101b147b4b22b2a57f8ce6fbf_SeqId_53_ - updating store file list. 2023-03-10 18:55:06,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/b3e5540101b147b4b22b2a57f8ce6fbf_SeqId_53_ into 75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:55:06,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__v2g2cnd2vl1pu8fu5qqc59bujrke3s4i6r7psag2e3fsef7akp2n10qjptkvb6n5/family1/19d657ed17a44a9f9ae6517dd9a8d8d9 into 75251ceb003f12a816b02e1f9c7cd641/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/b3e5540101b147b4b22b2a57f8ce6fbf_SeqId_53_) 2023-03-10 18:55:06,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__v2g2cnd2vl1pu8fu5qqc59bujrke3s4i6r7psag2e3fsef7akp2n10qjptkvb6n5/family1/19d657ed17a44a9f9ae6517dd9a8d8d9 2023-03-10 18:55:06,498 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:06,499 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:06,500 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120778ffd4af61f43f4b0c960bd50150920 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120778ffd4af61f43f4b0c960bd50150920 2023-03-10 18:55:06,501 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128ca7852135bda4dc3ae87573e350003cd to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128ca7852135bda4dc3ae87573e350003cd 2023-03-10 18:55:06,501 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-10 18:55:06,501 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-10 18:55:06,502 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:06,502 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:06,502 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:06,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:06,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:55:06,503 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-10 18:55:06,504 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 885 msec 2023-03-10 18:55:06,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-10 18:55:06,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:06,967 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-10 18:55:06,967 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-10 18:55:06,968 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-10 18:55:06,969 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-10 18:55:06,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, REOPEN/MOVE}] 2023-03-10 18:55:06,972 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, REOPEN/MOVE 2023-03-10 18:55:06,973 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=75251ceb003f12a816b02e1f9c7cd641, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:06,973 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678474506973"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474506973"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474506973"}]},"ts":"1678474506973"} 2023-03-10 18:55:06,974 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 75251ceb003f12a816b02e1f9c7cd641, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:07,125 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:07,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 75251ceb003f12a816b02e1f9c7cd641, disabling compactions & flushes 2023-03-10 18:55:07,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:07,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:07,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. after waiting 0 ms 2023-03-10 18:55:07,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:07,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-10 18:55:07,133 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:07,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:07,133 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 75251ceb003f12a816b02e1f9c7cd641 to self. 2023-03-10 18:55:07,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:07,135 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=75251ceb003f12a816b02e1f9c7cd641, regionState=CLOSED 2023-03-10 18:55:07,135 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678474507135"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474507135"}]},"ts":"1678474507135"} 2023-03-10 18:55:07,137 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-10 18:55:07,137 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 75251ceb003f12a816b02e1f9c7cd641, server=jenkins-hbase4.apache.org,43181,1678474414959 in 162 msec 2023-03-10 18:55:07,138 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=true 2023-03-10 18:55:07,288 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=75251ceb003f12a816b02e1f9c7cd641, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:07,288 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678474507288"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474507288"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474507288"}]},"ts":"1678474507288"} 2023-03-10 18:55:07,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 75251ceb003f12a816b02e1f9c7cd641, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:07,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:07,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 75251ceb003f12a816b02e1f9c7cd641, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:55:07,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:07,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:07,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:07,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:07,445 INFO [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:07,446 DEBUG [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:55:07,446 DEBUG [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1 2023-03-10 18:55:07,447 INFO [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75251ceb003f12a816b02e1f9c7cd641 columnFamilyName family1 2023-03-10 18:55:07,452 DEBUG [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/b3e5540101b147b4b22b2a57f8ce6fbf_SeqId_53_ 2023-03-10 18:55:07,454 DEBUG [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/family1/bf366c4e463749c18fe49a9f2de92afb 2023-03-10 18:55:07,455 INFO [StoreOpener-75251ceb003f12a816b02e1f9c7cd641-1] regionserver.HStore(310): Store=75251ceb003f12a816b02e1f9c7cd641/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:07,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:07,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:07,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:55:07,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 75251ceb003f12a816b02e1f9c7cd641; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74961037, jitterRate=0.11700649559497833}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:55:07,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:55:07,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641., pid=69, masterSystemTime=1678474507441 2023-03-10 18:55:07,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:07,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:55:07,461 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=75251ceb003f12a816b02e1f9c7cd641, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:07,461 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678474507461"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474507461"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474507461"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474507461"}]},"ts":"1678474507461"} 2023-03-10 18:55:07,463 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-10 18:55:07,463 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 75251ceb003f12a816b02e1f9c7cd641, server=jenkins-hbase4.apache.org,43181,1678474414959 in 172 msec 2023-03-10 18:55:07,465 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-10 18:55:07,465 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=75251ceb003f12a816b02e1f9c7cd641, REOPEN/MOVE in 491 msec 2023-03-10 18:55:07,466 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-10 18:55:07,466 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 496 msec 2023-03-10 18:55:07,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 912 msec 2023-03-10 18:55:07,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-10 18:55:07,470 INFO [Listener at localhost/36413] compactions.TestMobCompactor(278): alter status finished 2023-03-10 18:55:07,471 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-10 18:55:07,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-10 18:55:07,472 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-10 18:55:07,473 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:55:07,473 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475107473 2023-03-10 18:55:07,473 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:55:07,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-10 18:55:07,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:07,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x6cbd9a5e to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:55:07,479 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@8b0a4ab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:55:07,479 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:07,479 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:07,479 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:07,479 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:07,479 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:07,479 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:07,479 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:07,479 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-10 18:55:07,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-10 18:55:07,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:55:07,480 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-10 18:55:07,481 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 9 msec 2023-03-10 18:55:07,530 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=251 (was 242) - Thread LEAK? -, OpenFileDescriptor=642 (was 606) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=212 (was 225), ProcessCount=170 (was 170), AvailableMemoryMB=5572 (was 5589) 2023-03-10 18:55:07,540 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=251, OpenFileDescriptor=642, MaxFileDescriptor=60000, SystemLoadAverage=212, ProcessCount=170, AvailableMemoryMB=5571 2023-03-10 18:55:07,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-10 18:55:07,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-10 18:55:07,546 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:55:07,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-10 18:55:07,546 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:55:07,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-10 18:55:07,548 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:07,548 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2 empty. 2023-03-10 18:55:07,549 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:07,549 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-10 18:55:07,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-10 18:55:07,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-10 18:55:07,957 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-10 18:55:07,958 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => dea3b8ebf56eef3912ba21f1cd49e3e2, NAME => 'testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:55:08,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-10 18:55:08,364 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:08,364 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing dea3b8ebf56eef3912ba21f1cd49e3e2, disabling compactions & flushes 2023-03-10 18:55:08,364 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:08,364 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:08,365 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. after waiting 0 ms 2023-03-10 18:55:08,365 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:08,365 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:08,365 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for dea3b8ebf56eef3912ba21f1cd49e3e2: 2023-03-10 18:55:08,367 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:55:08,367 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678474508367"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474508367"}]},"ts":"1678474508367"} 2023-03-10 18:55:08,369 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-10 18:55:08,369 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:55:08,369 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474508369"}]},"ts":"1678474508369"} 2023-03-10 18:55:08,370 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-10 18:55:08,373 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=dea3b8ebf56eef3912ba21f1cd49e3e2, ASSIGN}] 2023-03-10 18:55:08,374 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=dea3b8ebf56eef3912ba21f1cd49e3e2, ASSIGN 2023-03-10 18:55:08,375 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=dea3b8ebf56eef3912ba21f1cd49e3e2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:55:08,526 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=dea3b8ebf56eef3912ba21f1cd49e3e2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:08,526 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678474508526"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474508526"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474508526"}]},"ts":"1678474508526"} 2023-03-10 18:55:08,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure dea3b8ebf56eef3912ba21f1cd49e3e2, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:08,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-10 18:55:08,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:08,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dea3b8ebf56eef3912ba21f1cd49e3e2, NAME => 'testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:55:08,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-10 18:55:08,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-10 18:55:08,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:08,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:08,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:08,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:08,685 INFO [StoreOpener-dea3b8ebf56eef3912ba21f1cd49e3e2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:08,686 DEBUG [StoreOpener-dea3b8ebf56eef3912ba21f1cd49e3e2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1 2023-03-10 18:55:08,687 DEBUG [StoreOpener-dea3b8ebf56eef3912ba21f1cd49e3e2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1 2023-03-10 18:55:08,687 INFO [StoreOpener-dea3b8ebf56eef3912ba21f1cd49e3e2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dea3b8ebf56eef3912ba21f1cd49e3e2 columnFamilyName f1 2023-03-10 18:55:08,688 INFO [StoreOpener-dea3b8ebf56eef3912ba21f1cd49e3e2-1] regionserver.HStore(310): Store=dea3b8ebf56eef3912ba21f1cd49e3e2/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:08,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:08,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:08,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:08,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:55:08,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dea3b8ebf56eef3912ba21f1cd49e3e2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60244082, jitterRate=-0.1022932231426239}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:55:08,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dea3b8ebf56eef3912ba21f1cd49e3e2: 2023-03-10 18:55:08,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2., pid=73, masterSystemTime=1678474508679 2023-03-10 18:55:08,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:08,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:08,696 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=dea3b8ebf56eef3912ba21f1cd49e3e2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:08,696 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678474508696"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474508696"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474508696"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474508696"}]},"ts":"1678474508696"} 2023-03-10 18:55:08,698 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-10 18:55:08,698 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure dea3b8ebf56eef3912ba21f1cd49e3e2, server=jenkins-hbase4.apache.org,43181,1678474414959 in 170 msec 2023-03-10 18:55:08,700 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-10 18:55:08,700 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=dea3b8ebf56eef3912ba21f1cd49e3e2, ASSIGN in 325 msec 2023-03-10 18:55:08,700 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:55:08,700 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474508700"}]},"ts":"1678474508700"} 2023-03-10 18:55:08,701 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-10 18:55:08,705 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:55:08,706 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1610 sec 2023-03-10 18:55:09,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-10 18:55:09,652 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-10 18:55:09,652 DEBUG [Listener at localhost/36413] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-10 18:55:09,654 INFO [Listener at localhost/36413] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-10 18:55:09,657 INFO [Listener at localhost/36413] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-10 18:55:09,658 INFO [Listener at localhost/36413] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-10 18:55:09,658 INFO [Listener at localhost/36413] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-10 18:55:09,663 INFO [Listener at localhost/36413] regionserver.HRegion(2745): Flushing dea3b8ebf56eef3912ba21f1cd49e3e2 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-10 18:55:10,082 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303104b39c9a46c674740a6d15adf88f9fd9a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303104b39c9a46c674740a6d15adf88f9fd9a 2023-03-10 18:55:10,488 INFO [Listener at localhost/36413] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/.tmp/f1/31ffb3fea0574aaa845a21f732b6a008 2023-03-10 18:55:10,492 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/.tmp/f1/31ffb3fea0574aaa845a21f732b6a008 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008 2023-03-10 18:55:10,496 INFO [Listener at localhost/36413] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008, entries=1, sequenceid=5, filesize=4.9 K 2023-03-10 18:55:10,497 INFO [Listener at localhost/36413] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for dea3b8ebf56eef3912ba21f1cd49e3e2 in 834ms, sequenceid=5, compaction requested=false 2023-03-10 18:55:10,497 DEBUG [Listener at localhost/36413] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-10 18:55:10,498 DEBUG [Listener at localhost/36413] regionserver.HRegion(2446): Flush status journal for dea3b8ebf56eef3912ba21f1cd49e3e2: 2023-03-10 18:55:10,500 INFO [Listener at localhost/36413] regionserver.HRegion(2745): Flushing dea3b8ebf56eef3912ba21f1cd49e3e2 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-10 18:55:10,819 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:55:10,856 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:55:10,856 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0fa6d5ca to 127.0.0.1:56431 2023-03-10 18:55:10,857 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:55:10,857 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:55:10,857 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5164c13c to 127.0.0.1:56431 2023-03-10 18:55:10,857 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:55:10,857 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:55:10,857 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x493ced45 to 127.0.0.1:56431 2023-03-10 18:55:10,857 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:55:10,857 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57d580ba to 127.0.0.1:56431 2023-03-10 18:55:10,857 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:55:10,858 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6cbd9a5e to 127.0.0.1:56431 2023-03-10 18:55:10,858 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:55:10,858 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:55:10,858 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5ed14a21 to 127.0.0.1:56431 2023-03-10 18:55:10,858 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:55:10,911 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303107b89ed52cd544d16aa749456870af6f1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303107b89ed52cd544d16aa749456870af6f1 2023-03-10 18:55:11,317 INFO [Listener at localhost/36413] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/.tmp/f1/dd03254365b34285a578c8566bb0a9a9 2023-03-10 18:55:11,321 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/.tmp/f1/dd03254365b34285a578c8566bb0a9a9 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9 2023-03-10 18:55:11,325 INFO [Listener at localhost/36413] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9, entries=1, sequenceid=9, filesize=4.9 K 2023-03-10 18:55:11,326 INFO [Listener at localhost/36413] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for dea3b8ebf56eef3912ba21f1cd49e3e2 in 826ms, sequenceid=9, compaction requested=false 2023-03-10 18:55:11,326 DEBUG [Listener at localhost/36413] regionserver.HRegion(2446): Flush status journal for dea3b8ebf56eef3912ba21f1cd49e3e2: 2023-03-10 18:55:11,328 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-10 18:55:11,329 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-10 18:55:11,330 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-10 18:55:11,330 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475111330 2023-03-10 18:55:11,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:55:11,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-10 18:55:11,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-10 18:55:11,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-10 18:55:11,332 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x1289d9e1 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:55:11,337 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1f11b2e7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:55:11,337 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230310 for table='testGetAfterCompaction' and column='f1' 2023-03-10 18:55:11,379 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,379 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,430 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,430 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,481 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,481 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,532 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,532 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,583 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,583 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,634 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,634 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,685 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,685 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,736 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,736 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,787 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,787 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,838 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,838 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,889 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,889 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,940 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,940 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,991 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:11,991 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,042 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,042 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,093 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,093 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,144 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,144 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,160 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031046470906f2df4319904210e7233f0131 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023031046470906f2df4319904210e7233f0131 2023-03-10 18:55:12,161 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230310 table='testGetAfterCompaction' column='f1' 2023-03-10 18:55:12,177 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:55:12,178 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39784, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:55:12,184 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:55:12,186 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44706, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:55:12,194 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230310/f1/fa66b482fbc248f9826df1b742c14339 first=Optional[r1] last=Optional[r2] 2023-03-10 18:55:12,194 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{f1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230310/f1/fa66b482fbc248f9826df1b742c14339}] 2023-03-10 18:55:12,195 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,195 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230310/f1/fa66b482fbc248f9826df1b742c14339 for inclusion in dea3b8ebf56eef3912ba21f1cd49e3e2/f1 2023-03-10 18:55:12,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-10 18:55:12,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:55:12,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for dea3b8ebf56eef3912ba21f1cd49e3e2: 2023-03-10 18:55:12,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230310/f1/fa66b482fbc248f9826df1b742c14339 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testGetAfterCompaction__3l6vcasrq51r053puphjs224hpj8unihmv6l952fjvnphbkt8pbvo631qqb4ei22/f1/fa66b482fbc248f9826df1b742c14339 2023-03-10 18:55:12,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testGetAfterCompaction__3l6vcasrq51r053puphjs224hpj8unihmv6l952fjvnphbkt8pbvo631qqb4ei22/f1/fa66b482fbc248f9826df1b742c14339 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_ 2023-03-10 18:55:12,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testGetAfterCompaction__3l6vcasrq51r053puphjs224hpj8unihmv6l952fjvnphbkt8pbvo631qqb4ei22/f1/fa66b482fbc248f9826df1b742c14339 into dea3b8ebf56eef3912ba21f1cd49e3e2/f1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_ - updating store file list. 2023-03-10 18:55:12,206 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_ into dea3b8ebf56eef3912ba21f1cd49e3e2/f1 2023-03-10 18:55:12,206 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testGetAfterCompaction__3l6vcasrq51r053puphjs224hpj8unihmv6l952fjvnphbkt8pbvo631qqb4ei22/f1/fa66b482fbc248f9826df1b742c14339 into dea3b8ebf56eef3912ba21f1cd49e3e2/f1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_) 2023-03-10 18:55:12,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testGetAfterCompaction__3l6vcasrq51r053puphjs224hpj8unihmv6l952fjvnphbkt8pbvo631qqb4ei22/f1/fa66b482fbc248f9826df1b742c14339 2023-03-10 18:55:12,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(1912): dea3b8ebf56eef3912ba21f1cd49e3e2/f1 is initiating minor compaction 2023-03-10 18:55:12,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6b9cdabe; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:55:12,207 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of dea3b8ebf56eef3912ba21f1cd49e3e2/f1 in testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:12,207 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/.tmp, totalSize=10.0 K 2023-03-10 18:55:12,207 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting dd03254365b34285a578c8566bb0a9a9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-10 18:55:12,207 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 7044ab99ee024569bb2354b1e816c688_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-10 18:55:12,209 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230310 table='testGetAfterCompaction' column='f1' 2023-03-10 18:55:12,209 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:12,211 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303104b39c9a46c674740a6d15adf88f9fd9a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303104b39c9a46c674740a6d15adf88f9fd9a 2023-03-10 18:55:12,212 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303107b89ed52cd544d16aa749456870af6f1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303107b89ed52cd544d16aa749456870af6f1 2023-03-10 18:55:12,212 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-10 18:55:12,212 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230310' 2023-03-10 18:55:12,213 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230310 for table='testGetAfterCompaction' and column='f1' 2023-03-10 18:55:12,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-10 18:55:12,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:55:12,214 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-10 18:55:12,215 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): dea3b8ebf56eef3912ba21f1cd49e3e2#f1#compaction#76 average throughput is 0.08 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 84.62 MB/second 2023-03-10 18:55:12,215 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 886 msec 2023-03-10 18:55:12,246 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,246 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,246 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,250 DEBUG [Listener at localhost/36413] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-10 18:55:12,253 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39798, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-10 18:55:12,254 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43181] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:12,254 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-03-10 18:55:12,254 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-10 18:55:12,254 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43181] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-10 18:55:12,254 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43181] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. because compaction request was cancelled 2023-03-10 18:55:12,306 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,306 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,306 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,357 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,357 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,357 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,408 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,408 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,408 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,459 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,459 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,459 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,510 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,510 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,510 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,560 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,561 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,561 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,611 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,611 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,612 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,662 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,662 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,662 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,713 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,713 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,713 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,764 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,764 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,764 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,815 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,815 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,815 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,866 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,866 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,866 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,917 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,917 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,917 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,968 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:12,968 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:12,968 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:13,019 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:13,019 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:13,019 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:13,030 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/.tmp/f1/8f75cf296c1949d096e40edc3bdde1fa as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/8f75cf296c1949d096e40edc3bdde1fa 2023-03-10 18:55:13,035 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in dea3b8ebf56eef3912ba21f1cd49e3e2/f1 of dea3b8ebf56eef3912ba21f1cd49e3e2 into 8f75cf296c1949d096e40edc3bdde1fa(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:55:13,035 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for dea3b8ebf56eef3912ba21f1cd49e3e2: 2023-03-10 18:55:13,035 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1678474512207; duration=0sec 2023-03-10 18:55:13,035 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:55:13,070 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:13,070 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678474512156; access_time=1678474511348; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-10 18:55:13,070 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/8f75cf296c1949d096e40edc3bdde1fa; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678474513025; access_time=1678474512211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:13,070 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474511317; access_time=1678474510501; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:13,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_] to archive 2023-03-10 18:55:13,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:13,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/dd03254365b34285a578c8566bb0a9a9 2023-03-10 18:55:13,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/7044ab99ee024569bb2354b1e816c688_SeqId_12_ 2023-03-10 18:55:13,121 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678474510487; access_time=1678474509666; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:13,121 INFO [Listener at localhost/36413] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/8f75cf296c1949d096e40edc3bdde1fa; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678474513025; access_time=1678474512211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-10 18:55:13,235 INFO [Listener at localhost/36413] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-10 18:55:13,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-03-10 18:55:13,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-10 18:55:13,251 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474513251"}]},"ts":"1678474513251"} 2023-03-10 18:55:13,253 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-10 18:55:13,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-10 18:55:13,256 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-10 18:55:13,258 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=dea3b8ebf56eef3912ba21f1cd49e3e2, UNASSIGN}] 2023-03-10 18:55:13,259 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=dea3b8ebf56eef3912ba21f1cd49e3e2, UNASSIGN 2023-03-10 18:55:13,259 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=dea3b8ebf56eef3912ba21f1cd49e3e2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:13,259 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678474513259"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474513259"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474513259"}]},"ts":"1678474513259"} 2023-03-10 18:55:13,261 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure dea3b8ebf56eef3912ba21f1cd49e3e2, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:13,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-10 18:55:13,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:13,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dea3b8ebf56eef3912ba21f1cd49e3e2, disabling compactions & flushes 2023-03-10 18:55:13,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:13,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:13,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. after waiting 0 ms 2023-03-10 18:55:13,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:13,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-10 18:55:13,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-10 18:55:13,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2. 2023-03-10 18:55:13,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dea3b8ebf56eef3912ba21f1cd49e3e2: 2023-03-10 18:55:13,420 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:13,420 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=dea3b8ebf56eef3912ba21f1cd49e3e2, regionState=CLOSED 2023-03-10 18:55:13,421 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678474513420"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474513420"}]},"ts":"1678474513420"} 2023-03-10 18:55:13,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-10 18:55:13,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure dea3b8ebf56eef3912ba21f1cd49e3e2, server=jenkins-hbase4.apache.org,43181,1678474414959 in 161 msec 2023-03-10 18:55:13,424 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-10 18:55:13,424 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=dea3b8ebf56eef3912ba21f1cd49e3e2, UNASSIGN in 165 msec 2023-03-10 18:55:13,424 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474513424"}]},"ts":"1678474513424"} 2023-03-10 18:55:13,425 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-10 18:55:13,427 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-10 18:55:13,428 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 185 msec 2023-03-10 18:55:13,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-10 18:55:13,556 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-10 18:55:13,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-03-10 18:55:13,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-10 18:55:13,571 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-10 18:55:13,572 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-10 18:55:13,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-10 18:55:13,575 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:13,577 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1, FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/recovered.edits] 2023-03-10 18:55:13,581 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/31ffb3fea0574aaa845a21f732b6a008 2023-03-10 18:55:13,582 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/8f75cf296c1949d096e40edc3bdde1fa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/f1/8f75cf296c1949d096e40edc3bdde1fa 2023-03-10 18:55:13,584 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/recovered.edits/15.seqid to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2/recovered.edits/15.seqid 2023-03-10 18:55:13,585 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testGetAfterCompaction/dea3b8ebf56eef3912ba21f1cd49e3e2 2023-03-10 18:55:13,585 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-10 18:55:13,585 DEBUG [PEWorker-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-10 18:55:13,586 DEBUG [PEWorker-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-10 18:55:13,589 DEBUG [PEWorker-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023031046470906f2df4319904210e7233f0131 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023031046470906f2df4319904210e7233f0131 2023-03-10 18:55:13,590 DEBUG [PEWorker-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-10 18:55:13,592 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-10 18:55:13,614 WARN [PEWorker-3] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-10 18:55:13,616 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-10 18:55:13,617 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-10 18:55:13,617 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-10 18:55:13,617 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678474513617"}]},"ts":"9223372036854775807"} 2023-03-10 18:55:13,619 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-10 18:55:13,619 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => dea3b8ebf56eef3912ba21f1cd49e3e2, NAME => 'testGetAfterCompaction,,1678474507543.dea3b8ebf56eef3912ba21f1cd49e3e2.', STARTKEY => '', ENDKEY => ''}] 2023-03-10 18:55:13,619 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-10 18:55:13,619 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1678474513619"}]},"ts":"9223372036854775807"} 2023-03-10 18:55:13,620 INFO [PEWorker-3] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-10 18:55:13,622 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-10 18:55:13,623 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 60 msec 2023-03-10 18:55:13,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-10 18:55:13,675 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-10 18:55:13,687 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=236 (was 251), OpenFileDescriptor=598 (was 642), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 212), ProcessCount=170 (was 170), AvailableMemoryMB=5531 (was 5571) 2023-03-10 18:55:13,698 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=236, OpenFileDescriptor=598, MaxFileDescriptor=60000, SystemLoadAverage=202, ProcessCount=170, AvailableMemoryMB=5530 2023-03-10 18:55:13,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-10 18:55:13,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-10 18:55:13,702 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:55:13,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-10 18:55:13,702 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:55:13,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-10 18:55:13,704 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706 2023-03-10 18:55:13,705 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:13,705 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:13,705 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706 empty. 2023-03-10 18:55:13,705 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89 empty. 2023-03-10 18:55:13,705 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf empty. 2023-03-10 18:55:13,705 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706 2023-03-10 18:55:13,706 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:13,706 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:13,706 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-10 18:55:13,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-10 18:55:14,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-10 18:55:14,120 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-10 18:55:14,122 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => b1d206b11bc34c2bde324dcef1ee6a89, NAME => 'testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:55:14,122 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => e46cb331129008feebe80244602e4706, NAME => 'testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:55:14,122 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => 7278512a19c794d2cb2655c58a8803cf, NAME => 'testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:55:14,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-10 18:55:14,539 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:14,539 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing b1d206b11bc34c2bde324dcef1ee6a89, disabling compactions & flushes 2023-03-10 18:55:14,539 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:14,539 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:14,539 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. after waiting 0 ms 2023-03-10 18:55:14,539 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:14,539 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:14,539 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:14,544 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:14,544 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:14,544 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing 7278512a19c794d2cb2655c58a8803cf, disabling compactions & flushes 2023-03-10 18:55:14,544 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing e46cb331129008feebe80244602e4706, disabling compactions & flushes 2023-03-10 18:55:14,544 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:14,544 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:14,544 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:14,544 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. after waiting 0 ms 2023-03-10 18:55:14,544 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:14,544 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:14,545 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:14,545 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:14,544 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. after waiting 0 ms 2023-03-10 18:55:14,545 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:14,545 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:14,545 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:14,547 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:55:14,548 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474514547"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474514547"}]},"ts":"1678474514547"} 2023-03-10 18:55:14,548 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474514547"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474514547"}]},"ts":"1678474514547"} 2023-03-10 18:55:14,548 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474514547"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474514547"}]},"ts":"1678474514547"} 2023-03-10 18:55:14,550 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-10 18:55:14,550 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:55:14,550 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474514550"}]},"ts":"1678474514550"} 2023-03-10 18:55:14,551 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-10 18:55:14,560 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e46cb331129008feebe80244602e4706, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b1d206b11bc34c2bde324dcef1ee6a89, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7278512a19c794d2cb2655c58a8803cf, ASSIGN}] 2023-03-10 18:55:14,561 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b1d206b11bc34c2bde324dcef1ee6a89, ASSIGN 2023-03-10 18:55:14,561 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e46cb331129008feebe80244602e4706, ASSIGN 2023-03-10 18:55:14,561 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7278512a19c794d2cb2655c58a8803cf, ASSIGN 2023-03-10 18:55:14,562 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b1d206b11bc34c2bde324dcef1ee6a89, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:55:14,562 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e46cb331129008feebe80244602e4706, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:55:14,562 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7278512a19c794d2cb2655c58a8803cf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:55:14,713 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=e46cb331129008feebe80244602e4706, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:14,713 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=b1d206b11bc34c2bde324dcef1ee6a89, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:14,713 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=7278512a19c794d2cb2655c58a8803cf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:14,714 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474514713"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474514713"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474514713"}]},"ts":"1678474514713"} 2023-03-10 18:55:14,714 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474514713"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474514713"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474514713"}]},"ts":"1678474514713"} 2023-03-10 18:55:14,714 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474514713"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474514713"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474514713"}]},"ts":"1678474514713"} 2023-03-10 18:55:14,715 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure e46cb331129008feebe80244602e4706, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:14,716 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=82, state=RUNNABLE; OpenRegionProcedure 7278512a19c794d2cb2655c58a8803cf, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:14,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=81, state=RUNNABLE; OpenRegionProcedure b1d206b11bc34c2bde324dcef1ee6a89, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:14,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-10 18:55:14,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:14,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b1d206b11bc34c2bde324dcef1ee6a89, NAME => 'testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.', STARTKEY => '1', ENDKEY => '2'} 2023-03-10 18:55:14,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:14,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:14,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:14,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:14,873 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:14,874 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1 2023-03-10 18:55:14,874 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1 2023-03-10 18:55:14,875 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b1d206b11bc34c2bde324dcef1ee6a89 columnFamilyName family1 2023-03-10 18:55:14,876 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(310): Store=b1d206b11bc34c2bde324dcef1ee6a89/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:14,876 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:14,877 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2 2023-03-10 18:55:14,877 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2 2023-03-10 18:55:14,877 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b1d206b11bc34c2bde324dcef1ee6a89 columnFamilyName family2 2023-03-10 18:55:14,878 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(310): Store=b1d206b11bc34c2bde324dcef1ee6a89/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:14,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:14,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:14,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:55:14,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:14,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:55:14,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b1d206b11bc34c2bde324dcef1ee6a89; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67557025, jitterRate=0.0066781193017959595}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:55:14,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:14,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89., pid=85, masterSystemTime=1678474514869 2023-03-10 18:55:14,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:14,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:14,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:14,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7278512a19c794d2cb2655c58a8803cf, NAME => 'testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.', STARTKEY => '2', ENDKEY => ''} 2023-03-10 18:55:14,886 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=b1d206b11bc34c2bde324dcef1ee6a89, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:14,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:14,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:14,886 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474514886"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474514886"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474514886"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474514886"}]},"ts":"1678474514886"} 2023-03-10 18:55:14,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:14,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:14,887 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:14,888 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1 2023-03-10 18:55:14,889 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1 2023-03-10 18:55:14,889 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=81 2023-03-10 18:55:14,889 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=81, state=SUCCESS; OpenRegionProcedure b1d206b11bc34c2bde324dcef1ee6a89, server=jenkins-hbase4.apache.org,43181,1678474414959 in 170 msec 2023-03-10 18:55:14,889 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7278512a19c794d2cb2655c58a8803cf columnFamilyName family1 2023-03-10 18:55:14,890 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(310): Store=7278512a19c794d2cb2655c58a8803cf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:14,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b1d206b11bc34c2bde324dcef1ee6a89, ASSIGN in 330 msec 2023-03-10 18:55:14,890 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:14,891 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2 2023-03-10 18:55:14,891 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2 2023-03-10 18:55:14,891 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7278512a19c794d2cb2655c58a8803cf columnFamilyName family2 2023-03-10 18:55:14,892 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(310): Store=7278512a19c794d2cb2655c58a8803cf/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:14,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:14,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:14,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:55:14,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:14,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:55:14,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7278512a19c794d2cb2655c58a8803cf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70712516, jitterRate=0.053698599338531494}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:55:14,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:14,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf., pid=84, masterSystemTime=1678474514869 2023-03-10 18:55:14,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:14,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:14,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:14,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e46cb331129008feebe80244602e4706, NAME => 'testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.', STARTKEY => '', ENDKEY => '1'} 2023-03-10 18:55:14,899 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=7278512a19c794d2cb2655c58a8803cf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:14,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink e46cb331129008feebe80244602e4706 2023-03-10 18:55:14,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:14,899 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474514899"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474514899"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474514899"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474514899"}]},"ts":"1678474514899"} 2023-03-10 18:55:14,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e46cb331129008feebe80244602e4706 2023-03-10 18:55:14,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e46cb331129008feebe80244602e4706 2023-03-10 18:55:14,900 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e46cb331129008feebe80244602e4706 2023-03-10 18:55:14,901 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1 2023-03-10 18:55:14,901 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=82 2023-03-10 18:55:14,901 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1 2023-03-10 18:55:14,901 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=82, state=SUCCESS; OpenRegionProcedure 7278512a19c794d2cb2655c58a8803cf, server=jenkins-hbase4.apache.org,43181,1678474414959 in 184 msec 2023-03-10 18:55:14,902 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e46cb331129008feebe80244602e4706 columnFamilyName family1 2023-03-10 18:55:14,902 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7278512a19c794d2cb2655c58a8803cf, ASSIGN in 341 msec 2023-03-10 18:55:14,903 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(310): Store=e46cb331129008feebe80244602e4706/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:14,903 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region e46cb331129008feebe80244602e4706 2023-03-10 18:55:14,904 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2 2023-03-10 18:55:14,904 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2 2023-03-10 18:55:14,905 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e46cb331129008feebe80244602e4706 columnFamilyName family2 2023-03-10 18:55:14,905 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(310): Store=e46cb331129008feebe80244602e4706/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:14,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706 2023-03-10 18:55:14,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706 2023-03-10 18:55:14,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:55:14,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e46cb331129008feebe80244602e4706 2023-03-10 18:55:14,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:55:14,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e46cb331129008feebe80244602e4706; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71796702, jitterRate=0.06985422968864441}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:55:14,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:14,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706., pid=83, masterSystemTime=1678474514869 2023-03-10 18:55:14,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:14,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:14,913 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=e46cb331129008feebe80244602e4706, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:14,913 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474514913"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474514913"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474514913"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474514913"}]},"ts":"1678474514913"} 2023-03-10 18:55:14,915 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-03-10 18:55:14,915 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure e46cb331129008feebe80244602e4706, server=jenkins-hbase4.apache.org,43181,1678474414959 in 199 msec 2023-03-10 18:55:14,917 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-03-10 18:55:14,917 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e46cb331129008feebe80244602e4706, ASSIGN in 356 msec 2023-03-10 18:55:14,917 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:55:14,917 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474514917"}]},"ts":"1678474514917"} 2023-03-10 18:55:14,918 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-10 18:55:14,920 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:55:14,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2200 sec 2023-03-10 18:55:15,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-10 18:55:15,809 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-10 18:55:15,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:15,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:15,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:15,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:15,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-10 18:55:15,821 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-10 18:55:15,821 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:15,821 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-10 18:55:15,821 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-10 18:55:15,822 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:15,822 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:15,823 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:15,823 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:15,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:15,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:15,823 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:15,823 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:15,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:15,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:15,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:15,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:15,824 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-10 18:55:15,824 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-10 18:55:15,824 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-10 18:55:15,825 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:15,825 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-10 18:55:15,825 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:15,825 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. started... 2023-03-10 18:55:15,825 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:15,825 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:55:15,826 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:15,826 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7278512a19c794d2cb2655c58a8803cf 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-10 18:55:15,826 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:15,825 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. started... 2023-03-10 18:55:15,826 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. started... 2023-03-10 18:55:15,826 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing e46cb331129008feebe80244602e4706 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-10 18:55:15,827 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b1d206b11bc34c2bde324dcef1ee6a89 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-10 18:55:15,849 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310676fd2889e2047939798d088407d6846 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310676fd2889e2047939798d088407d6846 2023-03-10 18:55:16,248 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303100110de7bf34f4f06b23c5bb30b8237aa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303100110de7bf34f4f06b23c5bb30b8237aa 2023-03-10 18:55:16,248 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310f0a1cc08ae174323961d9269bdffe719 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310f0a1cc08ae174323961d9269bdffe719 2023-03-10 18:55:16,258 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/960884e555924219b3af19c08d09a0fd 2023-03-10 18:55:16,653 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/751ae9dcd2a84cec82d96a4c3a837ccc 2023-03-10 18:55:16,653 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/72986f3f827c45f884d437495654452c 2023-03-10 18:55:16,670 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310d0c83762d79b4a66a1ea7b68a140b5eb to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310d0c83762d79b4a66a1ea7b68a140b5eb 2023-03-10 18:55:17,068 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310296c40995ed14d139df34998193d82f2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310296c40995ed14d139df34998193d82f2 2023-03-10 18:55:17,068 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303108d628aacc3b647beb3238c1d3beaa9d1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303108d628aacc3b647beb3238c1d3beaa9d1 2023-03-10 18:55:17,074 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/ac6f63f68c4f4410b8660234c52ecb68 2023-03-10 18:55:17,078 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/960884e555924219b3af19c08d09a0fd as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/960884e555924219b3af19c08d09a0fd 2023-03-10 18:55:17,081 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/960884e555924219b3af19c08d09a0fd, entries=4, sequenceid=5, filesize=5.4 K 2023-03-10 18:55:17,082 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/ac6f63f68c4f4410b8660234c52ecb68 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/ac6f63f68c4f4410b8660234c52ecb68 2023-03-10 18:55:17,085 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/ac6f63f68c4f4410b8660234c52ecb68, entries=2, sequenceid=5, filesize=5.1 K 2023-03-10 18:55:17,086 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for e46cb331129008feebe80244602e4706 in 1260ms, sequenceid=5, compaction requested=false 2023-03-10 18:55:17,086 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-10 18:55:17,087 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:17,087 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:17,087 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:55:17,473 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/d67d818c37f74d62a3582f788f5c5e93 2023-03-10 18:55:17,473 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/42edb717d3864e81adb8b53985567a5d 2023-03-10 18:55:17,478 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/751ae9dcd2a84cec82d96a4c3a837ccc as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/751ae9dcd2a84cec82d96a4c3a837ccc 2023-03-10 18:55:17,478 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/72986f3f827c45f884d437495654452c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/72986f3f827c45f884d437495654452c 2023-03-10 18:55:17,481 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/751ae9dcd2a84cec82d96a4c3a837ccc, entries=4, sequenceid=5, filesize=5.4 K 2023-03-10 18:55:17,481 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/42edb717d3864e81adb8b53985567a5d as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/42edb717d3864e81adb8b53985567a5d 2023-03-10 18:55:17,482 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/72986f3f827c45f884d437495654452c, entries=4, sequenceid=5, filesize=5.4 K 2023-03-10 18:55:17,483 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/d67d818c37f74d62a3582f788f5c5e93 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/d67d818c37f74d62a3582f788f5c5e93 2023-03-10 18:55:17,485 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/42edb717d3864e81adb8b53985567a5d, entries=2, sequenceid=5, filesize=5.1 K 2023-03-10 18:55:17,486 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 7278512a19c794d2cb2655c58a8803cf in 1660ms, sequenceid=5, compaction requested=false 2023-03-10 18:55:17,487 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:17,487 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:17,487 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:55:17,487 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/d67d818c37f74d62a3582f788f5c5e93, entries=2, sequenceid=5, filesize=5.1 K 2023-03-10 18:55:17,488 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for b1d206b11bc34c2bde324dcef1ee6a89 in 1661ms, sequenceid=5, compaction requested=false 2023-03-10 18:55:17,488 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:17,488 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-10 18:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-10 18:55:17,490 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:17,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:17,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:17,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:17,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:17,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:17,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:17,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-10 18:55:17,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d6030e5[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:17,492 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-10 18:55:17,492 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:17,494 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:17,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:17,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:17,494 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-10 18:55:17,494 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-10 18:55:17,494 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:17,494 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,494 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:17,496 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-10 18:55:17,496 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,496 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:17,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,496 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-10 18:55:17,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:17,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:17,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:17,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:17,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-10 18:55:17,498 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-10 18:55:17,498 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:17,498 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:17,498 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-10 18:55:17,498 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:17,499 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:17,499 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:17,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:17,499 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:17,500 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:17,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:17,501 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:17,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:17,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,503 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:17,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:17,504 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:17,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:17,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:17,504 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-10 18:55:17,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-10 18:55:17,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:17,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:17,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:17,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:17,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:17,504 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:17,504 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:17,504 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:17,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:17,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:17,604 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:17,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:17,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:17,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:17,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:17,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:17,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-10 18:55:17,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-10 18:55:17,611 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-10 18:55:17,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:17,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-10 18:55:17,611 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-10 18:55:17,612 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:17,612 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:17,615 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,615 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:17,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:17,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:17,615 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:17,615 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:17,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:17,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:17,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:17,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:17,616 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-10 18:55:17,616 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-10 18:55:17,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-10 18:55:17,617 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-10 18:55:17,617 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:17,617 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-10 18:55:17,621 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:17,621 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. started... 2023-03-10 18:55:17,621 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:17,621 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. started... 2023-03-10 18:55:17,621 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7278512a19c794d2cb2655c58a8803cf 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-10 18:55:17,621 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing e46cb331129008feebe80244602e4706 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-10 18:55:17,621 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:17,621 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. started... 2023-03-10 18:55:17,621 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:55:17,621 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b1d206b11bc34c2bde324dcef1ee6a89 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-10 18:55:17,621 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:18,048 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303107269d9f8e1c844ad81e42a5d32da0a16 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303107269d9f8e1c844ad81e42a5d32da0a16 2023-03-10 18:55:18,048 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310735dacb5b48d4a2faf383d9c602a19e4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310735dacb5b48d4a2faf383d9c602a19e4 2023-03-10 18:55:18,048 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310f498e39faa23437294854cc72b5586f3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230310f498e39faa23437294854cc72b5586f3 2023-03-10 18:55:18,453 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/926b09e4a5d841b4a7926d62a4656bef 2023-03-10 18:55:18,457 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/c264e685deff465a9024c52427e64a02 2023-03-10 18:55:18,457 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/de7cc23d176440b1a983425b09b04cf9 2023-03-10 18:55:18,472 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310e709b8bc9f0a4f7fa5e53a0d356f0096 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310e709b8bc9f0a4f7fa5e53a0d356f0096 2023-03-10 18:55:18,475 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310a80f64c7fc004d5fb660bb0a65021bd7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230310a80f64c7fc004d5fb660bb0a65021bd7 2023-03-10 18:55:18,871 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310989986e1fa9f4313b953f22e83644334 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310989986e1fa9f4313b953f22e83644334 2023-03-10 18:55:18,878 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/0c40ba5afb67448695dd46892c3b5bad 2023-03-10 18:55:18,880 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/26f6d99d4af945a7b769fe6ca46296a1 2023-03-10 18:55:18,882 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/926b09e4a5d841b4a7926d62a4656bef as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/926b09e4a5d841b4a7926d62a4656bef 2023-03-10 18:55:18,884 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/de7cc23d176440b1a983425b09b04cf9 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/de7cc23d176440b1a983425b09b04cf9 2023-03-10 18:55:18,885 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/926b09e4a5d841b4a7926d62a4656bef, entries=4, sequenceid=9, filesize=5.4 K 2023-03-10 18:55:18,886 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/0c40ba5afb67448695dd46892c3b5bad as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/0c40ba5afb67448695dd46892c3b5bad 2023-03-10 18:55:18,888 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/de7cc23d176440b1a983425b09b04cf9, entries=4, sequenceid=9, filesize=5.4 K 2023-03-10 18:55:18,888 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/26f6d99d4af945a7b769fe6ca46296a1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/26f6d99d4af945a7b769fe6ca46296a1 2023-03-10 18:55:18,889 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/0c40ba5afb67448695dd46892c3b5bad, entries=2, sequenceid=9, filesize=5.1 K 2023-03-10 18:55:18,890 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for b1d206b11bc34c2bde324dcef1ee6a89 in 1269ms, sequenceid=9, compaction requested=false 2023-03-10 18:55:18,890 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:18,890 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:18,890 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:55:18,892 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/26f6d99d4af945a7b769fe6ca46296a1, entries=2, sequenceid=9, filesize=5.1 K 2023-03-10 18:55:18,893 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 7278512a19c794d2cb2655c58a8803cf in 1272ms, sequenceid=9, compaction requested=false 2023-03-10 18:55:18,893 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:18,893 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:18,893 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:55:18,920 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:55:19,277 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/33c96e7f08cf4a1594f5c4700912b827 2023-03-10 18:55:19,282 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/c264e685deff465a9024c52427e64a02 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/c264e685deff465a9024c52427e64a02 2023-03-10 18:55:19,285 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/c264e685deff465a9024c52427e64a02, entries=4, sequenceid=9, filesize=5.4 K 2023-03-10 18:55:19,286 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/33c96e7f08cf4a1594f5c4700912b827 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/33c96e7f08cf4a1594f5c4700912b827 2023-03-10 18:55:19,290 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/33c96e7f08cf4a1594f5c4700912b827, entries=2, sequenceid=9, filesize=5.1 K 2023-03-10 18:55:19,290 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for e46cb331129008feebe80244602e4706 in 1669ms, sequenceid=9, compaction requested=false 2023-03-10 18:55:19,290 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:19,290 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:19,290 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:55:19,291 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:55:19,291 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,291 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-10 18:55:19,291 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-10 18:55:19,292 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:19,292 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:19,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:19,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:19,293 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:19,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:19,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:19,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:19,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:19,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-10 18:55:19,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27077d56[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:19,294 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-10 18:55:19,294 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:19,295 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:19,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:19,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:19,295 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-10 18:55:19,295 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-10 18:55:19,295 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:19,296 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,296 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:19,297 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-10 18:55:19,297 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:19,297 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,297 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-10 18:55:19,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:19,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:19,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:19,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:19,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:19,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:19,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:19,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:19,299 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-10 18:55:19,299 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-10 18:55:19,299 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:19,299 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:19,299 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-10 18:55:19,299 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:19,301 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:19,301 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:19,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:19,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:19,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:19,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:19,301 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:19,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:19,302 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:19,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:19,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:19,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:19,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:19,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:19,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:19,303 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:19,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:19,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,305 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,305 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:19,305 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:19,305 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:19,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:19,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:19,305 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:19,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-10 18:55:19,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:19,305 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:19,306 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-10 18:55:19,305 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:19,306 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:19,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:19,306 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:19,306 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:19,306 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:19,306 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:19,406 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:19,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:19,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:19,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:19,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:19,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:19,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-10 18:55:19,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-10 18:55:19,414 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-10 18:55:19,414 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:19,414 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-10 18:55:19,414 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-10 18:55:19,414 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:19,414 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:19,416 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:19,416 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:19,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:19,416 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:19,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:19,416 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:19,416 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:19,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:19,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:19,417 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-10 18:55:19,417 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-10 18:55:19,417 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-10 18:55:19,417 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-10 18:55:19,417 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:19,417 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-10 18:55:19,417 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:19,417 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. started... 2023-03-10 18:55:19,417 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:19,417 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7278512a19c794d2cb2655c58a8803cf 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-10 18:55:19,417 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:19,417 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. started... 2023-03-10 18:55:19,417 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. started... 2023-03-10 18:55:19,417 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:55:19,418 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:19,418 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing e46cb331129008feebe80244602e4706 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-10 18:55:19,418 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b1d206b11bc34c2bde324dcef1ee6a89 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-10 18:55:19,829 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310ff85a5ab8138412d8c34bd63cd8dd73f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310ff85a5ab8138412d8c34bd63cd8dd73f 2023-03-10 18:55:19,834 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310e4e3b007a2774a7ea06cdc9925ad5eef to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310e4e3b007a2774a7ea06cdc9925ad5eef 2023-03-10 18:55:19,834 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303106a98a00705914156b93966a1ed75b658 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303106a98a00705914156b93966a1ed75b658 2023-03-10 18:55:19,839 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/09edeadf05594761a6182f9e099f17c7 2023-03-10 18:55:20,238 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/b0ebc27232ba46888a2ac0e59dfff023 2023-03-10 18:55:20,239 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/972cf7551eb24a32b6db1f98c0be0339 2023-03-10 18:55:20,254 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303100f7afa40728a4d47a86963a9634d6f5a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303100f7afa40728a4d47a86963a9634d6f5a 2023-03-10 18:55:20,254 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310dc0207ddb2394e4491b9d68b2a2d7392 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310dc0207ddb2394e4491b9d68b2a2d7392 2023-03-10 18:55:20,258 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/6ab7ac83a5b54c48b8636359b183b0c5 2023-03-10 18:55:20,262 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/09edeadf05594761a6182f9e099f17c7 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/09edeadf05594761a6182f9e099f17c7 2023-03-10 18:55:20,265 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/09edeadf05594761a6182f9e099f17c7, entries=4, sequenceid=13, filesize=5.4 K 2023-03-10 18:55:20,266 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/6ab7ac83a5b54c48b8636359b183b0c5 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/6ab7ac83a5b54c48b8636359b183b0c5 2023-03-10 18:55:20,270 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/6ab7ac83a5b54c48b8636359b183b0c5, entries=2, sequenceid=13, filesize=5.1 K 2023-03-10 18:55:20,270 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 7278512a19c794d2cb2655c58a8803cf in 853ms, sequenceid=13, compaction requested=true 2023-03-10 18:55:20,270 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:20,271 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:20,271 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:55:20,652 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310cee5b82b7eea413db833498148b0cdd4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310cee5b82b7eea413db833498148b0cdd4 2023-03-10 18:55:20,659 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/74182c20bf50441cbffc56e7b1fbc79f 2023-03-10 18:55:20,662 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/972cf7551eb24a32b6db1f98c0be0339 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/972cf7551eb24a32b6db1f98c0be0339 2023-03-10 18:55:20,665 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/972cf7551eb24a32b6db1f98c0be0339, entries=4, sequenceid=13, filesize=5.4 K 2023-03-10 18:55:20,666 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/74182c20bf50441cbffc56e7b1fbc79f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/74182c20bf50441cbffc56e7b1fbc79f 2023-03-10 18:55:20,669 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/74182c20bf50441cbffc56e7b1fbc79f, entries=2, sequenceid=13, filesize=5.1 K 2023-03-10 18:55:20,670 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for e46cb331129008feebe80244602e4706 in 1252ms, sequenceid=13, compaction requested=true 2023-03-10 18:55:20,670 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:20,670 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:20,670 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:55:21,057 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/f4f5442a622542db857adf9fdfb3f80b 2023-03-10 18:55:21,062 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/b0ebc27232ba46888a2ac0e59dfff023 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/b0ebc27232ba46888a2ac0e59dfff023 2023-03-10 18:55:21,065 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/b0ebc27232ba46888a2ac0e59dfff023, entries=4, sequenceid=13, filesize=5.4 K 2023-03-10 18:55:21,066 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/f4f5442a622542db857adf9fdfb3f80b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/f4f5442a622542db857adf9fdfb3f80b 2023-03-10 18:55:21,069 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/f4f5442a622542db857adf9fdfb3f80b, entries=2, sequenceid=13, filesize=5.1 K 2023-03-10 18:55:21,070 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for b1d206b11bc34c2bde324dcef1ee6a89 in 1652ms, sequenceid=13, compaction requested=true 2023-03-10 18:55:21,070 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:21,070 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:21,070 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:55:21,070 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:55:21,070 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,070 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-10 18:55:21,070 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-10 18:55:21,072 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:21,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:21,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:21,072 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:21,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:21,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:21,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:21,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:21,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-10 18:55:21,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c200ef8[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:21,074 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-10 18:55:21,074 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:21,076 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:21,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:21,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:21,076 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-10 18:55:21,076 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-10 18:55:21,076 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:21,076 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,076 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:21,080 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,080 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-10 18:55:21,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:21,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:21,080 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:21,081 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-10 18:55:21,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:21,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:21,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:21,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:21,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:21,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:21,082 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-10 18:55:21,082 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:21,083 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:21,083 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-10 18:55:21,083 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:21,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-10 18:55:21,084 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:21,084 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:21,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:21,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:21,085 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:21,085 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:21,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:21,086 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:21,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:21,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:21,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,089 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,089 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:21,089 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:21,089 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,089 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:21,089 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:21,089 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:21,089 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:21,089 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-10 18:55:21,089 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:21,089 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:21,089 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:21,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:21,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:21,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-10 18:55:21,089 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:21,090 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:21,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:21,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:21,190 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:21,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:21,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:21,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:21,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:21,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:21,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-10 18:55:21,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-10 18:55:21,197 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-10 18:55:21,197 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:21,197 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-10 18:55:21,197 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-10 18:55:21,198 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:21,198 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:21,199 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:21,199 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:21,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:21,200 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:21,200 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:21,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:21,200 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:21,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:21,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:21,200 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-10 18:55:21,201 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-10 18:55:21,201 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-10 18:55:21,201 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-10 18:55:21,201 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:21,201 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-10 18:55:21,201 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:21,201 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. started... 2023-03-10 18:55:21,201 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:21,201 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. started... 2023-03-10 18:55:21,201 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7278512a19c794d2cb2655c58a8803cf 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-10 18:55:21,201 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing e46cb331129008feebe80244602e4706 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-10 18:55:21,202 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:55:21,202 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:21,202 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:21,202 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. started... 2023-03-10 18:55:21,202 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b1d206b11bc34c2bde324dcef1ee6a89 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-10 18:55:21,228 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303108865f8d67a9843bc90e3c0d40548a2cf to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303108865f8d67a9843bc90e3c0d40548a2cf 2023-03-10 18:55:21,622 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310c4a28d13bb2d42d18e7f25b50a5b7d73 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230310c4a28d13bb2d42d18e7f25b50a5b7d73 2023-03-10 18:55:21,624 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310094ac61681054c759b8da6cb2a2c5cfa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310094ac61681054c759b8da6cb2a2c5cfa 2023-03-10 18:55:21,629 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/49b0c444247a463e94f4df031173ec97 2023-03-10 18:55:21,637 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/0fce4462e53f4f3091ed9d3df87900d2 2023-03-10 18:55:21,643 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303108fca690ba6134eeaafd248f10f766b4b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303108fca690ba6134eeaafd248f10f766b4b 2023-03-10 18:55:22,028 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/6c02130ff1b7411db8858605388590dc 2023-03-10 18:55:22,047 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/9654f0864397433d9743a6560de07ee3 2023-03-10 18:55:22,052 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/49b0c444247a463e94f4df031173ec97 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/49b0c444247a463e94f4df031173ec97 2023-03-10 18:55:22,055 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/49b0c444247a463e94f4df031173ec97, entries=4, sequenceid=17, filesize=5.4 K 2023-03-10 18:55:22,056 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/9654f0864397433d9743a6560de07ee3 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/9654f0864397433d9743a6560de07ee3 2023-03-10 18:55:22,057 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303102c1c2f98400e4502a39d57fa32731a09 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303102c1c2f98400e4502a39d57fa32731a09 2023-03-10 18:55:22,060 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/9654f0864397433d9743a6560de07ee3, entries=2, sequenceid=17, filesize=5.1 K 2023-03-10 18:55:22,061 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for e46cb331129008feebe80244602e4706 in 860ms, sequenceid=17, compaction requested=true 2023-03-10 18:55:22,061 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:22,061 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:22,061 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:55:22,441 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303103c45b74a5cc045089803f7b3a22c87af to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303103c45b74a5cc045089803f7b3a22c87af 2023-03-10 18:55:22,462 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/4ced761ac1c0490980f2d865fa31a9e2 2023-03-10 18:55:22,466 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/0fce4462e53f4f3091ed9d3df87900d2 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/0fce4462e53f4f3091ed9d3df87900d2 2023-03-10 18:55:22,469 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/0fce4462e53f4f3091ed9d3df87900d2, entries=4, sequenceid=17, filesize=5.4 K 2023-03-10 18:55:22,469 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/4ced761ac1c0490980f2d865fa31a9e2 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/4ced761ac1c0490980f2d865fa31a9e2 2023-03-10 18:55:22,473 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/4ced761ac1c0490980f2d865fa31a9e2, entries=2, sequenceid=17, filesize=5.1 K 2023-03-10 18:55:22,473 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for b1d206b11bc34c2bde324dcef1ee6a89 in 1271ms, sequenceid=17, compaction requested=true 2023-03-10 18:55:22,473 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:22,473 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:22,474 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:55:22,847 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/daed0157d94b4114b52074154ee885d3 2023-03-10 18:55:22,852 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/6c02130ff1b7411db8858605388590dc as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/6c02130ff1b7411db8858605388590dc 2023-03-10 18:55:22,857 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/6c02130ff1b7411db8858605388590dc, entries=4, sequenceid=17, filesize=5.4 K 2023-03-10 18:55:22,857 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/daed0157d94b4114b52074154ee885d3 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/daed0157d94b4114b52074154ee885d3 2023-03-10 18:55:22,862 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/daed0157d94b4114b52074154ee885d3, entries=2, sequenceid=17, filesize=5.1 K 2023-03-10 18:55:22,863 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 7278512a19c794d2cb2655c58a8803cf in 1662ms, sequenceid=17, compaction requested=true 2023-03-10 18:55:22,863 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:22,863 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:22,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:55:22,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:55:22,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-10 18:55:22,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-10 18:55:22,865 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:22,865 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:22,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:22,865 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:22,865 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:22,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:22,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:22,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:22,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:22,867 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-10 18:55:22,867 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-10 18:55:22,867 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18bb90ab[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:22,867 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:22,868 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:22,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:22,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:22,868 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-10 18:55:22,868 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-10 18:55:22,868 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:22,868 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,868 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:22,871 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-10 18:55:22,871 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,871 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:22,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:22,871 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-10 18:55:22,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:22,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:22,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:22,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:22,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:22,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:22,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:22,873 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-10 18:55:22,873 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:22,873 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:22,873 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-10 18:55:22,873 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-10 18:55:22,873 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:22,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:22,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:22,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:22,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:22,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:22,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:22,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:22,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:22,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:22,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:22,876 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:22,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:22,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:22,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:22,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:22,877 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:22,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:22,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,879 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,880 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:22,880 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:22,880 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:22,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-10 18:55:22,880 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:22,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-10 18:55:22,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:22,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:22,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:22,880 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:22,880 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:22,880 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:22,880 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:22,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:22,880 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:22,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:22,880 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:22,880 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:22,981 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:22,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:22,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-10 18:55:22,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678474522992 (current time:1678474522992). 2023-03-10 18:55:22,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-10 18:55:22,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] zookeeper.ReadOnlyZKClient(139): Connect 0x7e2198f1 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:55:22,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1f1166e3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:55:23,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:55:23,006 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51886, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:55:23,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7e2198f1 to 127.0.0.1:56431 2023-03-10 18:55:23,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:55:23,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-10 18:55:23,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-10 18:55:23,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-10 18:55:23,017 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-10 18:55:23,018 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678475123018 2023-03-10 18:55:23,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-10 18:55:23,018 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1678474522982 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-10 18:55:23,019 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-10 18:55:23,020 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-10 18:55:23,020 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-03-10 18:55:23,021 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-10 18:55:23,022 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1678475123022 2023-03-10 18:55:23,024 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:23,024 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-10 18:55:23,124 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-10 18:55:23,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-10 18:55:23,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-10 18:55:23,128 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-10 18:55:23,328 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-10 18:55:23,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-10 18:55:23,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-10 18:55:23,329 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-10 18:55:23,434 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1678474522982 2023-03-10 18:55:23,434 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1678474522982' 2023-03-10 18:55:23,434 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-10 18:55:23,434 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1678474522982' starting 'acquire' 2023-03-10 18:55:23,434 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1678474522982', kicking off acquire phase on members. 2023-03-10 18:55:23,435 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678474522982 2023-03-10 18:55:23,435 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1678474522982 2023-03-10 18:55:23,436 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-10 18:55:23,436 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1678474522982/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:23,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-10 18:55:23,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-10 18:55:23,436 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1678474522982/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:23,436 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:23,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1678474522982 2023-03-10 18:55:23,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678474522982 2023-03-10 18:55:23,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-10 18:55:23,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1678474522982 2023-03-10 18:55:23,437 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1678474522982 from table testCompactionWithHFileLink type FLUSH 2023-03-10 18:55:23,439 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1678474522982 2023-03-10 18:55:23,439 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1678474522982' with timeout 300000ms 2023-03-10 18:55:23,439 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-10 18:55:23,439 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1678474522982' starting 'acquire' stage 2023-03-10 18:55:23,439 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1678474522982' locally acquired 2023-03-10 18:55:23,439 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (snaptb-1678474522982) in zk 2023-03-10 18:55:23,441 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1678474522982 2023-03-10 18:55:23,441 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678474522982/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:23,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1678474522982/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:23,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:23,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-10 18:55:23,441 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678474522982 2023-03-10 18:55:23,441 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1678474522982' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:23,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:23,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:23,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678474522982 2023-03-10 18:55:23,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:23,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:23,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'snaptb-1678474522982' on coordinator 2023-03-10 18:55:23,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34f07c41[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:23,442 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1678474522982' starting 'in-barrier' execution. 2023-03-10 18:55:23,443 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1678474522982 2023-03-10 18:55:23,444 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678474522982 2023-03-10 18:55:23,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1678474522982 2023-03-10 18:55:23,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1678474522982 2023-03-10 18:55:23,444 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1678474522982' received 'reached' from coordinator. 2023-03-10 18:55:23,444 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678474522982/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:23,444 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:23,445 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:23,445 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:23,445 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-10 18:55:23,445 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:23,445 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-10 18:55:23,445 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. started... 2023-03-10 18:55:23,445 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. started... 2023-03-10 18:55:23,445 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. started... 2023-03-10 18:55:23,445 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:23,445 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:23,445 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:23,446 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.' region-info for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,446 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.' region-info for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,446 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.' region-info for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,451 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-10 18:55:23,451 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-10 18:55:23,451 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-10 18:55:23,609 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:55:23,609 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1289d9e1 to 127.0.0.1:56431 2023-03-10 18:55:23,609 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-10 18:55:23,610 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:55:23,610 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/960884e555924219b3af19c08d09a0fd, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/c264e685deff465a9024c52427e64a02, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/972cf7551eb24a32b6db1f98c0be0339, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/49b0c444247a463e94f4df031173ec97] hfiles 2023-03-10 18:55:23,610 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/751ae9dcd2a84cec82d96a4c3a837ccc, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/de7cc23d176440b1a983425b09b04cf9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/09edeadf05594761a6182f9e099f17c7, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/6c02130ff1b7411db8858605388590dc] hfiles 2023-03-10 18:55:23,610 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/960884e555924219b3af19c08d09a0fd for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,610 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/72986f3f827c45f884d437495654452c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/926b09e4a5d841b4a7926d62a4656bef, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/b0ebc27232ba46888a2ac0e59dfff023, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/0fce4462e53f4f3091ed9d3df87900d2] hfiles 2023-03-10 18:55:23,610 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/72986f3f827c45f884d437495654452c for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,610 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/751ae9dcd2a84cec82d96a4c3a837ccc for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/c264e685deff465a9024c52427e64a02 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/926b09e4a5d841b4a7926d62a4656bef for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/972cf7551eb24a32b6db1f98c0be0339 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/b0ebc27232ba46888a2ac0e59dfff023 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/de7cc23d176440b1a983425b09b04cf9 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/0fce4462e53f4f3091ed9d3df87900d2 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/49b0c444247a463e94f4df031173ec97 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/09edeadf05594761a6182f9e099f17c7 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/ac6f63f68c4f4410b8660234c52ecb68, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/33c96e7f08cf4a1594f5c4700912b827, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/74182c20bf50441cbffc56e7b1fbc79f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/9654f0864397433d9743a6560de07ee3] hfiles 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/6c02130ff1b7411db8858605388590dc for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/ac6f63f68c4f4410b8660234c52ecb68 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/42edb717d3864e81adb8b53985567a5d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/26f6d99d4af945a7b769fe6ca46296a1, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/6ab7ac83a5b54c48b8636359b183b0c5, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/daed0157d94b4114b52074154ee885d3] hfiles 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/42edb717d3864e81adb8b53985567a5d for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/d67d818c37f74d62a3582f788f5c5e93, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/0c40ba5afb67448695dd46892c3b5bad, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/f4f5442a622542db857adf9fdfb3f80b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/4ced761ac1c0490980f2d865fa31a9e2] hfiles 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/d67d818c37f74d62a3582f788f5c5e93 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/26f6d99d4af945a7b769fe6ca46296a1 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,615 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/6ab7ac83a5b54c48b8636359b183b0c5 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,615 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/daed0157d94b4114b52074154ee885d3 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/33c96e7f08cf4a1594f5c4700912b827 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,614 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/0c40ba5afb67448695dd46892c3b5bad for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,615 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/74182c20bf50441cbffc56e7b1fbc79f for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,615 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/f4f5442a622542db857adf9fdfb3f80b for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,615 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/9654f0864397433d9743a6560de07ee3 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,615 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/4ced761ac1c0490980f2d865fa31a9e2 for snapshot=snaptb-1678474522982 2023-03-10 18:55:23,630 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-10 18:55:23,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-10 18:55:23,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-10 18:55:23,631 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-10 18:55:24,022 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. completed. 2023-03-10 18:55:24,022 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. completed. 2023-03-10 18:55:24,022 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:24,022 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:24,022 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. completed. 2023-03-10 18:55:24,022 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:24,022 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-10 18:55:24,022 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-10 18:55:24,022 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-10 18:55:24,023 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-10 18:55:24,023 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:24,023 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1678474522982' locally completed 2023-03-10 18:55:24,023 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1678474522982' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:24,025 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1678474522982' has notified controller of completion 2023-03-10 18:55:24,025 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678474522982/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:24,025 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:24,025 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1678474522982' completed. 2023-03-10 18:55:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1678474522982/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-10 18:55:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678474522982 2023-03-10 18:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678474522982 2023-03-10 18:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:24,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1678474522982' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:24,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'snaptb-1678474522982', counting down latch. Waiting for 0 more 2023-03-10 18:55:24,027 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1678474522982' execution completed 2023-03-10 18:55:24,027 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:24,027 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:24,027 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1678474522982 2023-03-10 18:55:24,027 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1678474522982including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-10 18:55:24,029 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678474522982 2023-03-10 18:55:24,029 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678474522982 2023-03-10 18:55:24,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1678474522982 2023-03-10 18:55:24,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1678474522982 2023-03-10 18:55:24,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678474522982 2023-03-10 18:55:24,029 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-10 18:55:24,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:24,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-10 18:55:24,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-10 18:55:24,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:24,029 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1678474522982/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:24,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-10 18:55:24,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678474522982 2023-03-10 18:55:24,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678474522982 2023-03-10 18:55:24,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:24,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678474522982 2023-03-10 18:55:24,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:24,030 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1678474522982/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:24,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:24,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678474522982 2023-03-10 18:55:24,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:24,032 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678474522982/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:24,032 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678474522982 2023-03-10 18:55:24,032 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-10 18:55:24,032 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678474522982 2023-03-10 18:55:24,032 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-10 18:55:24,032 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:24,032 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1678474522982 2023-03-10 18:55:24,032 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678474522982/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:24,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-10 18:55:24,033 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-10 18:55:24,033 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678474522982 2023-03-10 18:55:24,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-10 18:55:24,033 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1678474522982 2023-03-10 18:55:24,033 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678474522982 2023-03-10 18:55:24,033 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678474522982 2023-03-10 18:55:24,033 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-10 18:55:24,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-10 18:55:24,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310735dacb5b48d4a2faf383d9c602a19e4, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303108865f8d67a9843bc90e3c0d40548a2cf, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310f0a1cc08ae174323961d9269bdffe719, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310ff85a5ab8138412d8c34bd63cd8dd73f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303100110de7bf34f4f06b23c5bb30b8237aa, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303106a98a00705914156b93966a1ed75b658, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230310c4a28d13bb2d42d18e7f25b50a5b7d73, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230310f498e39faa23437294854cc72b5586f3, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310094ac61681054c759b8da6cb2a2c5cfa, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310676fd2889e2047939798d088407d6846, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303107269d9f8e1c844ad81e42a5d32da0a16, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310e4e3b007a2774a7ea06cdc9925ad5eef] mob files 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310735dacb5b48d4a2faf383d9c602a19e4 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303108865f8d67a9843bc90e3c0d40548a2cf 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310f0a1cc08ae174323961d9269bdffe719 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310ff85a5ab8138412d8c34bd63cd8dd73f 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303100110de7bf34f4f06b23c5bb30b8237aa 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303106a98a00705914156b93966a1ed75b658 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230310c4a28d13bb2d42d18e7f25b50a5b7d73 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230310f498e39faa23437294854cc72b5586f3 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310094ac61681054c759b8da6cb2a2c5cfa 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310676fd2889e2047939798d088407d6846 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303107269d9f8e1c844ad81e42a5d32da0a16 2023-03-10 18:55:24,035 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310e4e3b007a2774a7ea06cdc9925ad5eef 2023-03-10 18:55:24,036 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310296c40995ed14d139df34998193d82f2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303102c1c2f98400e4502a39d57fa32731a09, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310cee5b82b7eea413db833498148b0cdd4, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310e709b8bc9f0a4f7fa5e53a0d356f0096, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303100f7afa40728a4d47a86963a9634d6f5a, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303103c45b74a5cc045089803f7b3a22c87af, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303108d628aacc3b647beb3238c1d3beaa9d1, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230310a80f64c7fc004d5fb660bb0a65021bd7, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303108fca690ba6134eeaafd248f10f766b4b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310989986e1fa9f4313b953f22e83644334, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310d0c83762d79b4a66a1ea7b68a140b5eb, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310dc0207ddb2394e4491b9d68b2a2d7392] mob files 2023-03-10 18:55:24,036 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310296c40995ed14d139df34998193d82f2 2023-03-10 18:55:24,036 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303102c1c2f98400e4502a39d57fa32731a09 2023-03-10 18:55:24,036 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310cee5b82b7eea413db833498148b0cdd4 2023-03-10 18:55:24,036 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310e709b8bc9f0a4f7fa5e53a0d356f0096 2023-03-10 18:55:24,036 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303100f7afa40728a4d47a86963a9634d6f5a 2023-03-10 18:55:24,036 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303103c45b74a5cc045089803f7b3a22c87af 2023-03-10 18:55:24,036 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303108d628aacc3b647beb3238c1d3beaa9d1 2023-03-10 18:55:24,036 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230310a80f64c7fc004d5fb660bb0a65021bd7 2023-03-10 18:55:24,036 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303108fca690ba6134eeaafd248f10f766b4b 2023-03-10 18:55:24,037 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310989986e1fa9f4313b953f22e83644334 2023-03-10 18:55:24,037 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310d0c83762d79b4a66a1ea7b68a140b5eb 2023-03-10 18:55:24,037 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310dc0207ddb2394e4491b9d68b2a2d7392 2023-03-10 18:55:24,131 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-10 18:55:24,132 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-10 18:55:24,132 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-10 18:55:24,132 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-10 18:55:24,446 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1678474522982 2023-03-10 18:55:24,448 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.hbase-snapshot/.tmp/snaptb-1678474522982 2023-03-10 18:55:24,907 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.hbase-snapshot/.tmp/snaptb-1678474522982 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.hbase-snapshot/snaptb-1678474522982 2023-03-10 18:55:25,132 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-10 18:55:25,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-10 18:55:25,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-10 18:55:25,133 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-03-10 18:55:25,727 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1678474522982 of table testCompactionWithHFileLink completed 2023-03-10 18:55:25,727 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.hbase-snapshot/.tmp/snaptb-1678474522982 2023-03-10 18:55:25,728 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.hbase-snapshot/.tmp/snaptb-1678474522982 2023-03-10 18:55:25,728 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1678474522982 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678474523018Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1678474524446 (+1428 ms)Consolidate snapshot: snaptb-1678474522982 at 1678474524446Loading Region manifests for snaptb-1678474522982 at 1678474524446Writing data manifest for snaptb-1678474522982 at 1678474524464 (+18 ms)Verifying snapshot: snaptb-1678474522982 at 1678474524873 (+409 ms)Snapshot snaptb-1678474522982 of table testCompactionWithHFileLink completed at 1678474525727 (+854 ms) 2023-03-10 18:55:25,729 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-10 18:55:25,730 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.7090 sec 2023-03-10 18:55:27,133 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-10 18:55:27,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-10 18:55:27,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-10 18:55:27,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:27,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-10 18:55:27,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-10 18:55:27,164 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-10 18:55:27,164 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:27,164 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-10 18:55:27,164 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-10 18:55:27,164 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:27,164 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:27,168 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:27,168 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:27,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:27,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:27,169 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:27,169 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:27,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:27,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:27,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:27,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:27,169 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-10 18:55:27,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-10 18:55:27,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-10 18:55:27,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:27,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-10 18:55:27,170 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:27,170 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. started... 2023-03-10 18:55:27,170 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:27,170 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. started... 2023-03-10 18:55:27,170 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:27,170 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. started... 2023-03-10 18:55:27,170 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing e46cb331129008feebe80244602e4706 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-10 18:55:27,170 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b1d206b11bc34c2bde324dcef1ee6a89 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-10 18:55:27,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:55:27,170 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:27,170 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7278512a19c794d2cb2655c58a8803cf 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-10 18:55:27,588 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/cb6e45cd0ec34067bfa6bbd3c06a8f2d 2023-03-10 18:55:27,591 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb6e45cd0ec34067bfa6bbd3c06a8f2d 2023-03-10 18:55:27,987 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/7d615cd49fcc499ea352c16fc4694d05 2023-03-10 18:55:27,988 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/268b2bcc3b7349019528a605d5934ad2 2023-03-10 18:55:27,991 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7d615cd49fcc499ea352c16fc4694d05 2023-03-10 18:55:27,991 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 268b2bcc3b7349019528a605d5934ad2 2023-03-10 18:55:28,411 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/80539ae0c77c466d8be83ea36e223196 2023-03-10 18:55:28,415 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80539ae0c77c466d8be83ea36e223196 2023-03-10 18:55:28,416 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/cb6e45cd0ec34067bfa6bbd3c06a8f2d as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/cb6e45cd0ec34067bfa6bbd3c06a8f2d 2023-03-10 18:55:28,420 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb6e45cd0ec34067bfa6bbd3c06a8f2d 2023-03-10 18:55:28,420 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/cb6e45cd0ec34067bfa6bbd3c06a8f2d, entries=3, sequenceid=24, filesize=5.0 K 2023-03-10 18:55:28,421 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/80539ae0c77c466d8be83ea36e223196 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/80539ae0c77c466d8be83ea36e223196 2023-03-10 18:55:28,425 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80539ae0c77c466d8be83ea36e223196 2023-03-10 18:55:28,425 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/80539ae0c77c466d8be83ea36e223196, entries=1, sequenceid=24, filesize=5.0 K 2023-03-10 18:55:28,426 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for b1d206b11bc34c2bde324dcef1ee6a89 in 1256ms, sequenceid=24, compaction requested=true 2023-03-10 18:55:28,426 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:28,426 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:55:28,818 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/890645de201946dbacbf3ada3e563125 2023-03-10 18:55:28,822 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 890645de201946dbacbf3ada3e563125 2023-03-10 18:55:28,822 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/7d615cd49fcc499ea352c16fc4694d05 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/7d615cd49fcc499ea352c16fc4694d05 2023-03-10 18:55:28,825 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7d615cd49fcc499ea352c16fc4694d05 2023-03-10 18:55:28,825 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/7d615cd49fcc499ea352c16fc4694d05, entries=3, sequenceid=24, filesize=5.0 K 2023-03-10 18:55:28,826 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/4440e82d519448cbb0e28d06e05990f9 2023-03-10 18:55:28,826 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/890645de201946dbacbf3ada3e563125 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/890645de201946dbacbf3ada3e563125 2023-03-10 18:55:28,830 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4440e82d519448cbb0e28d06e05990f9 2023-03-10 18:55:28,830 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 890645de201946dbacbf3ada3e563125 2023-03-10 18:55:28,830 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/268b2bcc3b7349019528a605d5934ad2 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/268b2bcc3b7349019528a605d5934ad2 2023-03-10 18:55:28,830 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/890645de201946dbacbf3ada3e563125, entries=1, sequenceid=24, filesize=5.0 K 2023-03-10 18:55:28,831 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 7278512a19c794d2cb2655c58a8803cf in 1661ms, sequenceid=24, compaction requested=true 2023-03-10 18:55:28,831 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:28,831 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:28,831 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:55:28,833 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 268b2bcc3b7349019528a605d5934ad2 2023-03-10 18:55:28,833 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/268b2bcc3b7349019528a605d5934ad2, entries=3, sequenceid=24, filesize=5.0 K 2023-03-10 18:55:28,834 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/4440e82d519448cbb0e28d06e05990f9 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/4440e82d519448cbb0e28d06e05990f9 2023-03-10 18:55:28,837 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4440e82d519448cbb0e28d06e05990f9 2023-03-10 18:55:28,837 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/4440e82d519448cbb0e28d06e05990f9, entries=1, sequenceid=24, filesize=5.0 K 2023-03-10 18:55:28,837 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for e46cb331129008feebe80244602e4706 in 1667ms, sequenceid=24, compaction requested=true 2023-03-10 18:55:28,838 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:28,838 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:55:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:55:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-10 18:55:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-10 18:55:28,839 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,839 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:28,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:28,840 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:28,840 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:28,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:28,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-10 18:55:28,841 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-10 18:55:28,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28015a44[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:28,841 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:28,845 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:28,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:28,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:28,845 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-10 18:55:28,845 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-10 18:55:28,845 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:28,845 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,845 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:28,847 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-10 18:55:28,847 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,847 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:28,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:28,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:28,847 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-10 18:55:28,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:28,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:28,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:28,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:28,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:28,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:28,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-10 18:55:28,849 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-10 18:55:28,850 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:28,850 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:28,850 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-10 18:55:28,850 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:28,851 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:28,851 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:28,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:28,851 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:28,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:28,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:28,852 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:28,853 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-10 18:55:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,855 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,855 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:28,855 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:28,855 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:28,855 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:28,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:28,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-10 18:55:28,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:28,855 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-10 18:55:28,856 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:28,856 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:28,856 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-10 18:55:28,856 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-10 18:55:28,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-10 18:55:28,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:28,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:28,856 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:28,856 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:28,956 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:28,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:28,958 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:28,958 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 7278512a19c794d2cb2655c58a8803cf/family1 is initiating major compaction (all files) 2023-03-10 18:55:28,958 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:28,958 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:28,958 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 7278512a19c794d2cb2655c58a8803cf/family1 in testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:28,958 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/751ae9dcd2a84cec82d96a4c3a837ccc, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/de7cc23d176440b1a983425b09b04cf9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/09edeadf05594761a6182f9e099f17c7, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/6c02130ff1b7411db8858605388590dc, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/7d615cd49fcc499ea352c16fc4694d05] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp, totalSize=26.7 K 2023-03-10 18:55:28,959 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 751ae9dcd2a84cec82d96a4c3a837ccc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515817 2023-03-10 18:55:28,959 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting de7cc23d176440b1a983425b09b04cf9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517607 2023-03-10 18:55:28,960 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 09edeadf05594761a6182f9e099f17c7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:28,960 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 6c02130ff1b7411db8858605388590dc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521193 2023-03-10 18:55:28,960 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 7d615cd49fcc499ea352c16fc4694d05, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-10 18:55:28,972 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 7278512a19c794d2cb2655c58a8803cf#family1#compaction#107 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:29,381 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303100d8d89c450f547eda4387f15cc5c23f5_del 2023-03-10 18:55:29,381 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303100d8d89c450f547eda4387f15cc5c23f5_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303100d8d89c450f547eda4387f15cc5c23f5_del 2023-03-10 18:55:30,194 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a6fa1de5ba504799a7ad832729a2f94d 2023-03-10 18:55:30,195 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/a6fa1de5ba504799a7ad832729a2f94d as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a6fa1de5ba504799a7ad832729a2f94d 2023-03-10 18:55:30,199 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a6fa1de5ba504799a7ad832729a2f94d 2023-03-10 18:55:30,200 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7278512a19c794d2cb2655c58a8803cf/family1 of 7278512a19c794d2cb2655c58a8803cf into a6fa1de5ba504799a7ad832729a2f94d(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:30,200 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:30,200 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:30,200 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 7278512a19c794d2cb2655c58a8803cf/family2 is initiating major compaction (all files) 2023-03-10 18:55:30,200 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:30,200 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:30,200 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 7278512a19c794d2cb2655c58a8803cf/family2 in testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:30,200 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/42edb717d3864e81adb8b53985567a5d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/26f6d99d4af945a7b769fe6ca46296a1, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/6ab7ac83a5b54c48b8636359b183b0c5, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/daed0157d94b4114b52074154ee885d3, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/890645de201946dbacbf3ada3e563125] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp, totalSize=25.4 K 2023-03-10 18:55:30,201 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 42edb717d3864e81adb8b53985567a5d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515817 2023-03-10 18:55:30,201 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 26f6d99d4af945a7b769fe6ca46296a1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517607 2023-03-10 18:55:30,201 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 6ab7ac83a5b54c48b8636359b183b0c5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:30,202 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting daed0157d94b4114b52074154ee885d3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521193 2023-03-10 18:55:30,202 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 890645de201946dbacbf3ada3e563125, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-10 18:55:30,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/751ae9dcd2a84cec82d96a4c3a837ccc, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/de7cc23d176440b1a983425b09b04cf9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/09edeadf05594761a6182f9e099f17c7, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/6c02130ff1b7411db8858605388590dc, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/7d615cd49fcc499ea352c16fc4694d05] to archive 2023-03-10 18:55:30,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:30,213 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/751ae9dcd2a84cec82d96a4c3a837ccc to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/751ae9dcd2a84cec82d96a4c3a837ccc 2023-03-10 18:55:30,214 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 7278512a19c794d2cb2655c58a8803cf#family2#compaction#108 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:30,214 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/de7cc23d176440b1a983425b09b04cf9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/de7cc23d176440b1a983425b09b04cf9 2023-03-10 18:55:30,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/09edeadf05594761a6182f9e099f17c7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/09edeadf05594761a6182f9e099f17c7 2023-03-10 18:55:30,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/6c02130ff1b7411db8858605388590dc to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/6c02130ff1b7411db8858605388590dc 2023-03-10 18:55:30,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/7d615cd49fcc499ea352c16fc4694d05 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/7d615cd49fcc499ea352c16fc4694d05 2023-03-10 18:55:30,221 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230310bcde23db410844c482a3ec3804da69ea_del 2023-03-10 18:55:30,222 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310bcde23db410844c482a3ec3804da69ea_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230310bcde23db410844c482a3ec3804da69ea_del 2023-03-10 18:55:31,037 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 665093e9bec34f3c819fb572256f905a 2023-03-10 18:55:31,038 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/665093e9bec34f3c819fb572256f905a as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/665093e9bec34f3c819fb572256f905a 2023-03-10 18:55:31,041 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 665093e9bec34f3c819fb572256f905a 2023-03-10 18:55:31,042 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7278512a19c794d2cb2655c58a8803cf/family2 of 7278512a19c794d2cb2655c58a8803cf into 665093e9bec34f3c819fb572256f905a(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:55:31,042 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:31,043 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:31,043 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): e46cb331129008feebe80244602e4706/family1 is initiating major compaction (all files) 2023-03-10 18:55:31,043 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:31,043 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:31,043 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of e46cb331129008feebe80244602e4706/family1 in testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:31,043 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/960884e555924219b3af19c08d09a0fd, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/c264e685deff465a9024c52427e64a02, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/972cf7551eb24a32b6db1f98c0be0339, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/49b0c444247a463e94f4df031173ec97, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/268b2bcc3b7349019528a605d5934ad2] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp, totalSize=26.7 K 2023-03-10 18:55:31,044 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 960884e555924219b3af19c08d09a0fd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515815 2023-03-10 18:55:31,044 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting c264e685deff465a9024c52427e64a02, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517606 2023-03-10 18:55:31,044 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 972cf7551eb24a32b6db1f98c0be0339, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:31,045 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 49b0c444247a463e94f4df031173ec97, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521192 2023-03-10 18:55:31,045 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 268b2bcc3b7349019528a605d5934ad2, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-10 18:55:31,055 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): e46cb331129008feebe80244602e4706#family1#compaction#109 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:31,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/42edb717d3864e81adb8b53985567a5d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/26f6d99d4af945a7b769fe6ca46296a1, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/6ab7ac83a5b54c48b8636359b183b0c5, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/daed0157d94b4114b52074154ee885d3, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/890645de201946dbacbf3ada3e563125] to archive 2023-03-10 18:55:31,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:31,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/42edb717d3864e81adb8b53985567a5d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/42edb717d3864e81adb8b53985567a5d 2023-03-10 18:55:31,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/26f6d99d4af945a7b769fe6ca46296a1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/26f6d99d4af945a7b769fe6ca46296a1 2023-03-10 18:55:31,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/6ab7ac83a5b54c48b8636359b183b0c5 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/6ab7ac83a5b54c48b8636359b183b0c5 2023-03-10 18:55:31,118 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/daed0157d94b4114b52074154ee885d3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/daed0157d94b4114b52074154ee885d3 2023-03-10 18:55:31,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/890645de201946dbacbf3ada3e563125 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/890645de201946dbacbf3ada3e563125 2023-03-10 18:55:31,465 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023031014bf81fb910c4e67b27a8cef27071368_del 2023-03-10 18:55:31,465 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031014bf81fb910c4e67b27a8cef27071368_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031014bf81fb910c4e67b27a8cef27071368_del 2023-03-10 18:55:32,279 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 87ef6b48495d41f582df0e67ae23864d 2023-03-10 18:55:32,280 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/87ef6b48495d41f582df0e67ae23864d as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/87ef6b48495d41f582df0e67ae23864d 2023-03-10 18:55:32,283 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 87ef6b48495d41f582df0e67ae23864d 2023-03-10 18:55:32,284 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e46cb331129008feebe80244602e4706/family1 of e46cb331129008feebe80244602e4706 into 87ef6b48495d41f582df0e67ae23864d(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:32,284 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:32,284 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:32,284 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): e46cb331129008feebe80244602e4706/family2 is initiating major compaction (all files) 2023-03-10 18:55:32,284 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:32,284 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:32,285 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of e46cb331129008feebe80244602e4706/family2 in testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:32,285 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/ac6f63f68c4f4410b8660234c52ecb68, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/33c96e7f08cf4a1594f5c4700912b827, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/74182c20bf50441cbffc56e7b1fbc79f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/9654f0864397433d9743a6560de07ee3, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/4440e82d519448cbb0e28d06e05990f9] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp, totalSize=25.4 K 2023-03-10 18:55:32,285 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting ac6f63f68c4f4410b8660234c52ecb68, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515815 2023-03-10 18:55:32,285 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 33c96e7f08cf4a1594f5c4700912b827, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517606 2023-03-10 18:55:32,286 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 74182c20bf50441cbffc56e7b1fbc79f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:32,286 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 9654f0864397433d9743a6560de07ee3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521192 2023-03-10 18:55:32,286 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 4440e82d519448cbb0e28d06e05990f9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-10 18:55:32,297 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): e46cb331129008feebe80244602e4706#family2#compaction#110 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:32,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/960884e555924219b3af19c08d09a0fd, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/c264e685deff465a9024c52427e64a02, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/972cf7551eb24a32b6db1f98c0be0339, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/49b0c444247a463e94f4df031173ec97, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/268b2bcc3b7349019528a605d5934ad2] to archive 2023-03-10 18:55:32,311 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:32,312 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/960884e555924219b3af19c08d09a0fd to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/960884e555924219b3af19c08d09a0fd 2023-03-10 18:55:32,313 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/c264e685deff465a9024c52427e64a02 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/c264e685deff465a9024c52427e64a02 2023-03-10 18:55:32,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/972cf7551eb24a32b6db1f98c0be0339 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/972cf7551eb24a32b6db1f98c0be0339 2023-03-10 18:55:32,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/49b0c444247a463e94f4df031173ec97 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/49b0c444247a463e94f4df031173ec97 2023-03-10 18:55:32,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/268b2bcc3b7349019528a605d5934ad2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/268b2bcc3b7349019528a605d5934ad2 2023-03-10 18:55:32,707 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230310355d8c4bf863464e9cefeb20e4255132_del 2023-03-10 18:55:32,707 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310355d8c4bf863464e9cefeb20e4255132_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310355d8c4bf863464e9cefeb20e4255132_del 2023-03-10 18:55:33,522 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2a544eab33ee44328dd4bb97fcd2828b 2023-03-10 18:55:33,523 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/2a544eab33ee44328dd4bb97fcd2828b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/2a544eab33ee44328dd4bb97fcd2828b 2023-03-10 18:55:33,526 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2a544eab33ee44328dd4bb97fcd2828b 2023-03-10 18:55:33,527 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e46cb331129008feebe80244602e4706/family2 of e46cb331129008feebe80244602e4706 into 2a544eab33ee44328dd4bb97fcd2828b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:33,527 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:33,527 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:33,527 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): b1d206b11bc34c2bde324dcef1ee6a89/family1 is initiating major compaction (all files) 2023-03-10 18:55:33,527 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:33,527 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:33,527 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of b1d206b11bc34c2bde324dcef1ee6a89/family1 in testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:33,528 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/72986f3f827c45f884d437495654452c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/926b09e4a5d841b4a7926d62a4656bef, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/b0ebc27232ba46888a2ac0e59dfff023, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/0fce4462e53f4f3091ed9d3df87900d2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/cb6e45cd0ec34067bfa6bbd3c06a8f2d] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp, totalSize=26.7 K 2023-03-10 18:55:33,528 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 72986f3f827c45f884d437495654452c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515816 2023-03-10 18:55:33,528 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 926b09e4a5d841b4a7926d62a4656bef, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517606 2023-03-10 18:55:33,528 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting b0ebc27232ba46888a2ac0e59dfff023, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:33,529 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 0fce4462e53f4f3091ed9d3df87900d2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521192 2023-03-10 18:55:33,529 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting cb6e45cd0ec34067bfa6bbd3c06a8f2d, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-10 18:55:33,538 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): b1d206b11bc34c2bde324dcef1ee6a89#family1#compaction#111 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:33,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/ac6f63f68c4f4410b8660234c52ecb68, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/33c96e7f08cf4a1594f5c4700912b827, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/74182c20bf50441cbffc56e7b1fbc79f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/9654f0864397433d9743a6560de07ee3, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/4440e82d519448cbb0e28d06e05990f9] to archive 2023-03-10 18:55:33,615 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:33,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/ac6f63f68c4f4410b8660234c52ecb68 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/ac6f63f68c4f4410b8660234c52ecb68 2023-03-10 18:55:33,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/33c96e7f08cf4a1594f5c4700912b827 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/33c96e7f08cf4a1594f5c4700912b827 2023-03-10 18:55:33,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/74182c20bf50441cbffc56e7b1fbc79f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/74182c20bf50441cbffc56e7b1fbc79f 2023-03-10 18:55:33,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/9654f0864397433d9743a6560de07ee3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/9654f0864397433d9743a6560de07ee3 2023-03-10 18:55:33,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/4440e82d519448cbb0e28d06e05990f9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/4440e82d519448cbb0e28d06e05990f9 2023-03-10 18:55:33,945 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230310fa0ab34a53764e1fb79a33a2bbf6676f_del 2023-03-10 18:55:33,945 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310fa0ab34a53764e1fb79a33a2bbf6676f_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310fa0ab34a53764e1fb79a33a2bbf6676f_del 2023-03-10 18:55:34,760 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f82ca8420d24d279b297e05e3af3f69 2023-03-10 18:55:34,761 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/8f82ca8420d24d279b297e05e3af3f69 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/8f82ca8420d24d279b297e05e3af3f69 2023-03-10 18:55:34,765 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f82ca8420d24d279b297e05e3af3f69 2023-03-10 18:55:34,766 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b1d206b11bc34c2bde324dcef1ee6a89/family1 of b1d206b11bc34c2bde324dcef1ee6a89 into 8f82ca8420d24d279b297e05e3af3f69(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:34,766 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:34,766 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:34,766 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): b1d206b11bc34c2bde324dcef1ee6a89/family2 is initiating major compaction (all files) 2023-03-10 18:55:34,766 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:34,766 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:34,767 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of b1d206b11bc34c2bde324dcef1ee6a89/family2 in testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:34,767 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/d67d818c37f74d62a3582f788f5c5e93, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/0c40ba5afb67448695dd46892c3b5bad, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/f4f5442a622542db857adf9fdfb3f80b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/4ced761ac1c0490980f2d865fa31a9e2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/80539ae0c77c466d8be83ea36e223196] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp, totalSize=25.4 K 2023-03-10 18:55:34,768 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting d67d818c37f74d62a3582f788f5c5e93, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515816 2023-03-10 18:55:34,768 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 0c40ba5afb67448695dd46892c3b5bad, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517606 2023-03-10 18:55:34,769 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting f4f5442a622542db857adf9fdfb3f80b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:34,769 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 4ced761ac1c0490980f2d865fa31a9e2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521192 2023-03-10 18:55:34,769 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 80539ae0c77c466d8be83ea36e223196, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-10 18:55:34,781 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): b1d206b11bc34c2bde324dcef1ee6a89#family2#compaction#112 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:34,812 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/72986f3f827c45f884d437495654452c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/926b09e4a5d841b4a7926d62a4656bef, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/b0ebc27232ba46888a2ac0e59dfff023, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/0fce4462e53f4f3091ed9d3df87900d2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/cb6e45cd0ec34067bfa6bbd3c06a8f2d] to archive 2023-03-10 18:55:34,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:34,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/72986f3f827c45f884d437495654452c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/72986f3f827c45f884d437495654452c 2023-03-10 18:55:34,816 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/926b09e4a5d841b4a7926d62a4656bef to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/926b09e4a5d841b4a7926d62a4656bef 2023-03-10 18:55:34,817 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/b0ebc27232ba46888a2ac0e59dfff023 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/b0ebc27232ba46888a2ac0e59dfff023 2023-03-10 18:55:34,818 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/0fce4462e53f4f3091ed9d3df87900d2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/0fce4462e53f4f3091ed9d3df87900d2 2023-03-10 18:55:34,818 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/cb6e45cd0ec34067bfa6bbd3c06a8f2d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/cb6e45cd0ec34067bfa6bbd3c06a8f2d 2023-03-10 18:55:35,189 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303108070b35dcd4b4b53bd45a7e47b2ac200_del 2023-03-10 18:55:35,189 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303108070b35dcd4b4b53bd45a7e47b2ac200_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303108070b35dcd4b4b53bd45a7e47b2ac200_del 2023-03-10 18:55:36,003 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e317c4aad04b4825b1d7567b55bd59fa 2023-03-10 18:55:36,004 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/e317c4aad04b4825b1d7567b55bd59fa as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/e317c4aad04b4825b1d7567b55bd59fa 2023-03-10 18:55:36,007 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e317c4aad04b4825b1d7567b55bd59fa 2023-03-10 18:55:36,008 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b1d206b11bc34c2bde324dcef1ee6a89/family2 of b1d206b11bc34c2bde324dcef1ee6a89 into e317c4aad04b4825b1d7567b55bd59fa(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:36,008 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:36,011 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/d67d818c37f74d62a3582f788f5c5e93, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/0c40ba5afb67448695dd46892c3b5bad, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/f4f5442a622542db857adf9fdfb3f80b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/4ced761ac1c0490980f2d865fa31a9e2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/80539ae0c77c466d8be83ea36e223196] to archive 2023-03-10 18:55:36,012 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:36,014 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/d67d818c37f74d62a3582f788f5c5e93 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/d67d818c37f74d62a3582f788f5c5e93 2023-03-10 18:55:36,014 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:36,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/0c40ba5afb67448695dd46892c3b5bad to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/0c40ba5afb67448695dd46892c3b5bad 2023-03-10 18:55:36,017 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/f4f5442a622542db857adf9fdfb3f80b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/f4f5442a622542db857adf9fdfb3f80b 2023-03-10 18:55:36,018 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/4ced761ac1c0490980f2d865fa31a9e2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/4ced761ac1c0490980f2d865fa31a9e2 2023-03-10 18:55:36,019 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:36,020 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/80539ae0c77c466d8be83ea36e223196 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/80539ae0c77c466d8be83ea36e223196 2023-03-10 18:55:36,024 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:36,025 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:36,031 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-10 18:55:36,066 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-10 18:55:36,069 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023031014bf81fb910c4e67b27a8cef27071368_del 2023-03-10 18:55:36,073 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230310fa0ab34a53764e1fb79a33a2bbf6676f_del 2023-03-10 18:55:36,076 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303100d8d89c450f547eda4387f15cc5c23f5_del 2023-03-10 18:55:36,076 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:36,076 DEBUG [Listener at localhost/36413] zookeeper.ReadOnlyZKClient(139): Connect 0x6b3a96c8 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:55:36,082 DEBUG [Listener at localhost/36413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49d301c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:55:36,082 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:36,082 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:36,082 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:36,147 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031087b32fcb24684735927cae60c11a58d3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031087b32fcb24684735927cae60c11a58d3 2023-03-10 18:55:36,148 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:36,162 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:55:36,166 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41716, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:55:36,173 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:55:36,175 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34240, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:55:36,186 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230310/family1/b6c2ee75b2a94a9ea5bdda22a5abb386 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-10 18:55:36,187 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230310/family1/b6c2ee75b2a94a9ea5bdda22a5abb386}] 2023-03-10 18:55:36,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230310/family1/b6c2ee75b2a94a9ea5bdda22a5abb386 for inclusion in b1d206b11bc34c2bde324dcef1ee6a89/family1 2023-03-10 18:55:36,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-10 18:55:36,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-10 18:55:36,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:36,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230310/family1/b6c2ee75b2a94a9ea5bdda22a5abb386 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__sdalhuqo5qfve21khe0g9gfb95eqp2ep51lvphblifk7gkqsbiaulir8r4ag3irc/family1/b6c2ee75b2a94a9ea5bdda22a5abb386 2023-03-10 18:55:36,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__sdalhuqo5qfve21khe0g9gfb95eqp2ep51lvphblifk7gkqsbiaulir8r4ag3irc/family1/b6c2ee75b2a94a9ea5bdda22a5abb386 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/1f773ee681ad4010bca9a4d59c208c9f_SeqId_29_ 2023-03-10 18:55:36,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__sdalhuqo5qfve21khe0g9gfb95eqp2ep51lvphblifk7gkqsbiaulir8r4ag3irc/family1/b6c2ee75b2a94a9ea5bdda22a5abb386 into b1d206b11bc34c2bde324dcef1ee6a89/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/1f773ee681ad4010bca9a4d59c208c9f_SeqId_29_ - updating store file list. 2023-03-10 18:55:36,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/1f773ee681ad4010bca9a4d59c208c9f_SeqId_29_ into b1d206b11bc34c2bde324dcef1ee6a89/family1 2023-03-10 18:55:36,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__sdalhuqo5qfve21khe0g9gfb95eqp2ep51lvphblifk7gkqsbiaulir8r4ag3irc/family1/b6c2ee75b2a94a9ea5bdda22a5abb386 into b1d206b11bc34c2bde324dcef1ee6a89/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/1f773ee681ad4010bca9a4d59c208c9f_SeqId_29_) 2023-03-10 18:55:36,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__sdalhuqo5qfve21khe0g9gfb95eqp2ep51lvphblifk7gkqsbiaulir8r4ag3irc/family1/b6c2ee75b2a94a9ea5bdda22a5abb386 2023-03-10 18:55:36,203 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:36,203 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:36,204 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310735dacb5b48d4a2faf383d9c602a19e4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310735dacb5b48d4a2faf383d9c602a19e4 2023-03-10 18:55:36,205 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303108865f8d67a9843bc90e3c0d40548a2cf to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303108865f8d67a9843bc90e3c0d40548a2cf 2023-03-10 18:55:36,206 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310f0a1cc08ae174323961d9269bdffe719 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310f0a1cc08ae174323961d9269bdffe719 2023-03-10 18:55:36,207 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310ff85a5ab8138412d8c34bd63cd8dd73f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310ff85a5ab8138412d8c34bd63cd8dd73f 2023-03-10 18:55:36,207 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-10 18:55:36,207 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230310' 2023-03-10 18:55:36,207 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:36,566 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303102963c250214548829d6b7cb4f0ecec34 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303102963c250214548829d6b7cb4f0ecec34 2023-03-10 18:55:36,567 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:36,590 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230310/family1/c667a043253147eb95708816bc1e9179 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-10 18:55:36,592 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230310/family1/c667a043253147eb95708816bc1e9179}] 2023-03-10 18:55:36,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230310/family1/c667a043253147eb95708816bc1e9179 for inclusion in e46cb331129008feebe80244602e4706/family1 2023-03-10 18:55:36,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-10 18:55:36,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last=1 2023-03-10 18:55:36,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:36,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230310/family1/c667a043253147eb95708816bc1e9179 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__og9m8i4d2f2tfrtdtmmgcd9sn2dp02tg53unp70d48o4dcibvg38ao0rl2s8jeru/family1/c667a043253147eb95708816bc1e9179 2023-03-10 18:55:36,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__og9m8i4d2f2tfrtdtmmgcd9sn2dp02tg53unp70d48o4dcibvg38ao0rl2s8jeru/family1/c667a043253147eb95708816bc1e9179 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/7bdfb969c6c74cc596889d4168e818f0_SeqId_29_ 2023-03-10 18:55:36,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__og9m8i4d2f2tfrtdtmmgcd9sn2dp02tg53unp70d48o4dcibvg38ao0rl2s8jeru/family1/c667a043253147eb95708816bc1e9179 into e46cb331129008feebe80244602e4706/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/7bdfb969c6c74cc596889d4168e818f0_SeqId_29_ - updating store file list. 2023-03-10 18:55:36,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/7bdfb969c6c74cc596889d4168e818f0_SeqId_29_ into e46cb331129008feebe80244602e4706/family1 2023-03-10 18:55:36,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__og9m8i4d2f2tfrtdtmmgcd9sn2dp02tg53unp70d48o4dcibvg38ao0rl2s8jeru/family1/c667a043253147eb95708816bc1e9179 into e46cb331129008feebe80244602e4706/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/7bdfb969c6c74cc596889d4168e818f0_SeqId_29_) 2023-03-10 18:55:36,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__og9m8i4d2f2tfrtdtmmgcd9sn2dp02tg53unp70d48o4dcibvg38ao0rl2s8jeru/family1/c667a043253147eb95708816bc1e9179 2023-03-10 18:55:36,602 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:36,603 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:36,604 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310094ac61681054c759b8da6cb2a2c5cfa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310094ac61681054c759b8da6cb2a2c5cfa 2023-03-10 18:55:36,605 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310676fd2889e2047939798d088407d6846 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310676fd2889e2047939798d088407d6846 2023-03-10 18:55:36,606 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303107269d9f8e1c844ad81e42a5d32da0a16 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303107269d9f8e1c844ad81e42a5d32da0a16 2023-03-10 18:55:36,607 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310e4e3b007a2774a7ea06cdc9925ad5eef to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230310e4e3b007a2774a7ea06cdc9925ad5eef 2023-03-10 18:55:36,607 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-10 18:55:36,607 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230310' 2023-03-10 18:55:36,607 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:36,931 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023031016b7d936a45145d699d46c8a1695d72c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023031016b7d936a45145d699d46c8a1695d72c 2023-03-10 18:55:36,932 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:36,963 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230310/family1/a7d76150dbea49f2ad027b080e1de716 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-10 18:55:36,963 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230310/family1/a7d76150dbea49f2ad027b080e1de716}] 2023-03-10 18:55:36,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230310/family1/a7d76150dbea49f2ad027b080e1de716 for inclusion in 7278512a19c794d2cb2655c58a8803cf/family1 2023-03-10 18:55:36,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-10 18:55:36,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-10 18:55:36,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:36,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230310/family1/a7d76150dbea49f2ad027b080e1de716 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__ckh5phpeaaf2d326od4euv3bjrj2f37cc75on84sonedn44urd1sg1ll6l9all7k/family1/a7d76150dbea49f2ad027b080e1de716 2023-03-10 18:55:36,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__ckh5phpeaaf2d326od4euv3bjrj2f37cc75on84sonedn44urd1sg1ll6l9all7k/family1/a7d76150dbea49f2ad027b080e1de716 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a57cddc5664247f2b980ab3e59f716e9_SeqId_29_ 2023-03-10 18:55:36,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__ckh5phpeaaf2d326od4euv3bjrj2f37cc75on84sonedn44urd1sg1ll6l9all7k/family1/a7d76150dbea49f2ad027b080e1de716 into 7278512a19c794d2cb2655c58a8803cf/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a57cddc5664247f2b980ab3e59f716e9_SeqId_29_ - updating store file list. 2023-03-10 18:55:36,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a57cddc5664247f2b980ab3e59f716e9_SeqId_29_ into 7278512a19c794d2cb2655c58a8803cf/family1 2023-03-10 18:55:36,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__ckh5phpeaaf2d326od4euv3bjrj2f37cc75on84sonedn44urd1sg1ll6l9all7k/family1/a7d76150dbea49f2ad027b080e1de716 into 7278512a19c794d2cb2655c58a8803cf/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a57cddc5664247f2b980ab3e59f716e9_SeqId_29_) 2023-03-10 18:55:36,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__ckh5phpeaaf2d326od4euv3bjrj2f37cc75on84sonedn44urd1sg1ll6l9all7k/family1/a7d76150dbea49f2ad027b080e1de716 2023-03-10 18:55:36,977 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:36,977 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:36,978 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303100110de7bf34f4f06b23c5bb30b8237aa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303100110de7bf34f4f06b23c5bb30b8237aa 2023-03-10 18:55:36,979 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303106a98a00705914156b93966a1ed75b658 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303106a98a00705914156b93966a1ed75b658 2023-03-10 18:55:36,980 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230310c4a28d13bb2d42d18e7f25b50a5b7d73 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230310c4a28d13bb2d42d18e7f25b50a5b7d73 2023-03-10 18:55:36,981 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230310f498e39faa23437294854cc72b5586f3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230310f498e39faa23437294854cc72b5586f3 2023-03-10 18:55:36,981 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-10 18:55:36,981 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230310' 2023-03-10 18:55:36,981 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:36,981 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:36,981 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:36,981 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031014bf81fb910c4e67b27a8cef27071368_del] 2023-03-10 18:55:36,982 DEBUG [Listener at localhost/36413] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:36,983 DEBUG [Listener at localhost/36413] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031014bf81fb910c4e67b27a8cef27071368_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031014bf81fb910c4e67b27a8cef27071368_del 2023-03-10 18:55:36,983 INFO [Listener at localhost/36413] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-10 18:55:36,983 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310fa0ab34a53764e1fb79a33a2bbf6676f_del] 2023-03-10 18:55:36,983 DEBUG [Listener at localhost/36413] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:36,984 DEBUG [Listener at localhost/36413] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310fa0ab34a53764e1fb79a33a2bbf6676f_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310fa0ab34a53764e1fb79a33a2bbf6676f_del 2023-03-10 18:55:36,984 INFO [Listener at localhost/36413] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-10 18:55:36,984 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303100d8d89c450f547eda4387f15cc5c23f5_del] 2023-03-10 18:55:36,985 DEBUG [Listener at localhost/36413] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:36,986 DEBUG [Listener at localhost/36413] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303100d8d89c450f547eda4387f15cc5c23f5_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303100d8d89c450f547eda4387f15cc5c23f5_del 2023-03-10 18:55:36,986 INFO [Listener at localhost/36413] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-10 18:55:36,988 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:36,990 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:36,993 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:36,994 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:36,999 INFO [Listener at localhost/36413] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-10 18:55:36,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-03-10 18:55:37,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-10 18:55:37,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-10 18:55:37,002 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474537002"}]},"ts":"1678474537002"} 2023-03-10 18:55:37,004 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-10 18:55:37,006 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-10 18:55:37,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e46cb331129008feebe80244602e4706, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b1d206b11bc34c2bde324dcef1ee6a89, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7278512a19c794d2cb2655c58a8803cf, UNASSIGN}] 2023-03-10 18:55:37,007 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7278512a19c794d2cb2655c58a8803cf, UNASSIGN 2023-03-10 18:55:37,008 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b1d206b11bc34c2bde324dcef1ee6a89, UNASSIGN 2023-03-10 18:55:37,008 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e46cb331129008feebe80244602e4706, UNASSIGN 2023-03-10 18:55:37,008 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=7278512a19c794d2cb2655c58a8803cf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:37,008 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=b1d206b11bc34c2bde324dcef1ee6a89, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:37,008 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=e46cb331129008feebe80244602e4706, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:37,008 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474537008"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474537008"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474537008"}]},"ts":"1678474537008"} 2023-03-10 18:55:37,008 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474537008"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474537008"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474537008"}]},"ts":"1678474537008"} 2023-03-10 18:55:37,008 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474537008"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474537008"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474537008"}]},"ts":"1678474537008"} 2023-03-10 18:55:37,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure b1d206b11bc34c2bde324dcef1ee6a89, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:37,010 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=91, state=RUNNABLE; CloseRegionProcedure 7278512a19c794d2cb2655c58a8803cf, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:37,010 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=89, state=RUNNABLE; CloseRegionProcedure e46cb331129008feebe80244602e4706, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:37,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-10 18:55:37,161 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:37,161 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b1d206b11bc34c2bde324dcef1ee6a89, disabling compactions & flushes 2023-03-10 18:55:37,161 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:37,161 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:37,161 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. after waiting 0 ms 2023-03-10 18:55:37,161 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:37,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-10 18:55:37,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:37,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:37,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:37,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:37,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7278512a19c794d2cb2655c58a8803cf, disabling compactions & flushes 2023-03-10 18:55:37,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:37,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:37,168 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=b1d206b11bc34c2bde324dcef1ee6a89, regionState=CLOSED 2023-03-10 18:55:37,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. after waiting 0 ms 2023-03-10 18:55:37,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:37,168 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474537168"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474537168"}]},"ts":"1678474537168"} 2023-03-10 18:55:37,171 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-03-10 18:55:37,171 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure b1d206b11bc34c2bde324dcef1ee6a89, server=jenkins-hbase4.apache.org,43181,1678474414959 in 160 msec 2023-03-10 18:55:37,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b1d206b11bc34c2bde324dcef1ee6a89, UNASSIGN in 165 msec 2023-03-10 18:55:37,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-10 18:55:37,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:37,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:37,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:37,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e46cb331129008feebe80244602e4706 2023-03-10 18:55:37,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e46cb331129008feebe80244602e4706, disabling compactions & flushes 2023-03-10 18:55:37,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:37,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:37,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. after waiting 0 ms 2023-03-10 18:55:37,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:37,174 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=7278512a19c794d2cb2655c58a8803cf, regionState=CLOSED 2023-03-10 18:55:37,175 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474537174"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474537174"}]},"ts":"1678474537174"} 2023-03-10 18:55:37,177 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=91 2023-03-10 18:55:37,177 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=91, state=SUCCESS; CloseRegionProcedure 7278512a19c794d2cb2655c58a8803cf, server=jenkins-hbase4.apache.org,43181,1678474414959 in 166 msec 2023-03-10 18:55:37,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-10 18:55:37,180 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7278512a19c794d2cb2655c58a8803cf, UNASSIGN in 171 msec 2023-03-10 18:55:37,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:37,182 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:37,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e46cb331129008feebe80244602e4706 2023-03-10 18:55:37,183 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=e46cb331129008feebe80244602e4706, regionState=CLOSED 2023-03-10 18:55:37,183 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474537183"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474537183"}]},"ts":"1678474537183"} 2023-03-10 18:55:37,185 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=89 2023-03-10 18:55:37,186 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=89, state=SUCCESS; CloseRegionProcedure e46cb331129008feebe80244602e4706, server=jenkins-hbase4.apache.org,43181,1678474414959 in 174 msec 2023-03-10 18:55:37,187 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-03-10 18:55:37,187 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e46cb331129008feebe80244602e4706, UNASSIGN in 180 msec 2023-03-10 18:55:37,187 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474537187"}]},"ts":"1678474537187"} 2023-03-10 18:55:37,188 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-10 18:55:37,190 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-10 18:55:37,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 190 msec 2023-03-10 18:55:37,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-10 18:55:37,305 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-10 18:55:37,314 INFO [Listener at localhost/36413] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1678474522982-1678474537314 2023-03-10 18:55:37,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-10 18:55:37,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678474537316 (current time:1678474537316). 2023-03-10 18:55:37,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-10 18:55:37,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] zookeeper.ReadOnlyZKClient(139): Connect 0x435be4b0 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:55:37,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@fa49797, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:55:37,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:55:37,325 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41732, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:55:37,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x435be4b0 to 127.0.0.1:56431 2023-03-10 18:55:37,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:55:37,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-10 18:55:37,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-10 18:55:37,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-10 18:55:37,329 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-10 18:55:37,330 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678475137330 2023-03-10 18:55:37,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-10 18:55:37,330 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1678474522982-1678474537314 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-10 18:55:37,330 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:37,330 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-10 18:55:37,430 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-10 18:55:37,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-10 18:55:37,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-10 18:55:37,432 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-10 18:55:37,632 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-10 18:55:37,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-10 18:55:37,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-10 18:55:37,633 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-10 18:55:37,736 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-10 18:55:37,738 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-10 18:55:37,738 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-10 18:55:37,738 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-10 18:55:37,738 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-10 18:55:37,738 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-10 18:55:37,738 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-10 18:55:37,738 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-10 18:55:37,738 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a57cddc5664247f2b980ab3e59f716e9_SeqId_29_, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a6fa1de5ba504799a7ad832729a2f94d] hfiles 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/7bdfb969c6c74cc596889d4168e818f0_SeqId_29_, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/87ef6b48495d41f582df0e67ae23864d] hfiles 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/1f773ee681ad4010bca9a4d59c208c9f_SeqId_29_, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/8f82ca8420d24d279b297e05e3af3f69] hfiles 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a57cddc5664247f2b980ab3e59f716e9_SeqId_29_ 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/1f773ee681ad4010bca9a4d59c208c9f_SeqId_29_ 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a6fa1de5ba504799a7ad832729a2f94d 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/8f82ca8420d24d279b297e05e3af3f69 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/7bdfb969c6c74cc596889d4168e818f0_SeqId_29_ 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031087b32fcb24684735927cae60c11a58d3, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023031016b7d936a45145d699d46c8a1695d72c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303102963c250214548829d6b7cb4f0ecec34] hfiles 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/87ef6b48495d41f582df0e67ae23864d 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031087b32fcb24684735927cae60c11a58d3 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023031016b7d936a45145d699d46c8a1695d72c 2023-03-10 18:55:37,741 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303102963c250214548829d6b7cb4f0ecec34 2023-03-10 18:55:37,742 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/665093e9bec34f3c819fb572256f905a] hfiles 2023-03-10 18:55:37,742 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/665093e9bec34f3c819fb572256f905a 2023-03-10 18:55:37,742 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/2a544eab33ee44328dd4bb97fcd2828b] hfiles 2023-03-10 18:55:37,742 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/2a544eab33ee44328dd4bb97fcd2828b 2023-03-10 18:55:37,742 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/e317c4aad04b4825b1d7567b55bd59fa] hfiles 2023-03-10 18:55:37,742 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/e317c4aad04b4825b1d7567b55bd59fa 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310296c40995ed14d139df34998193d82f2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303102c1c2f98400e4502a39d57fa32731a09, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303108070b35dcd4b4b53bd45a7e47b2ac200_del, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310cee5b82b7eea413db833498148b0cdd4, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310e709b8bc9f0a4f7fa5e53a0d356f0096, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303100f7afa40728a4d47a86963a9634d6f5a, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303103c45b74a5cc045089803f7b3a22c87af, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303108d628aacc3b647beb3238c1d3beaa9d1, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230310a80f64c7fc004d5fb660bb0a65021bd7, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230310bcde23db410844c482a3ec3804da69ea_del, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310355d8c4bf863464e9cefeb20e4255132_del, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303108fca690ba6134eeaafd248f10f766b4b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310989986e1fa9f4313b953f22e83644334, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310d0c83762d79b4a66a1ea7b68a140b5eb, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310dc0207ddb2394e4491b9d68b2a2d7392] hfiles 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310296c40995ed14d139df34998193d82f2 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303102c1c2f98400e4502a39d57fa32731a09 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303108070b35dcd4b4b53bd45a7e47b2ac200_del 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310cee5b82b7eea413db833498148b0cdd4 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230310e709b8bc9f0a4f7fa5e53a0d356f0096 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303100f7afa40728a4d47a86963a9634d6f5a 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303103c45b74a5cc045089803f7b3a22c87af 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303108d628aacc3b647beb3238c1d3beaa9d1 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230310a80f64c7fc004d5fb660bb0a65021bd7 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230310bcde23db410844c482a3ec3804da69ea_del 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310355d8c4bf863464e9cefeb20e4255132_del 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303108fca690ba6134eeaafd248f10f766b4b 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310989986e1fa9f4313b953f22e83644334 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310d0c83762d79b4a66a1ea7b68a140b5eb 2023-03-10 18:55:37,748 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310dc0207ddb2394e4491b9d68b2a2d7392 2023-03-10 18:55:37,933 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-10 18:55:37,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-10 18:55:37,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-10 18:55:37,934 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-10 18:55:38,152 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-10 18:55:38,153 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1678474522982-1678474537314 2023-03-10 18:55:38,153 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678474522982-1678474537314 2023-03-10 18:55:38,434 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-10 18:55:38,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-10 18:55:38,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-10 18:55:38,436 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-10 18:55:38,575 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678474522982-1678474537314 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.hbase-snapshot/hbase-failsafe-snaptb-1678474522982-1678474537314 2023-03-10 18:55:39,388 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1678474522982-1678474537314 of table testCompactionWithHFileLink completed 2023-03-10 18:55:39,389 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678474522982-1678474537314 2023-03-10 18:55:39,389 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678474522982-1678474537314 2023-03-10 18:55:39,389 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1678474522982-1678474537314 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678474537330Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1678474537736 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1678474522982-1678474537314 at 1678474538153 (+417 ms)Loading Region manifests for hbase-failsafe-snaptb-1678474522982-1678474537314 at 1678474538153Writing data manifest for hbase-failsafe-snaptb-1678474522982-1678474537314 at 1678474538156 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1678474522982-1678474537314 at 1678474538562 (+406 ms)Snapshot hbase-failsafe-snaptb-1678474522982-1678474537314 of table testCompactionWithHFileLink completed at 1678474539388 (+826 ms) 2023-03-10 18:55:39,390 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-10 18:55:39,391 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0620 sec 2023-03-10 18:55:39,436 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-10 18:55:39,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-10 18:55:39,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1678474522982-1678474537314 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-10 18:55:39,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678474522982" table: "testCompactionWithHFileLink" creation_time: 1678474522992 type: FLUSH version: 2 ttl: 0 ) 2023-03-10 18:55:39,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1678474522982 as table=testCompactionWithHFileLink 2023-03-10 18:55:39,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-10 18:55:39,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-10 18:55:39,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-10 18:55:39,890 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-10 18:55:39,891 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-10 18:55:39,891 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-10 18:55:39,892 INFO [PEWorker-4] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-10 18:55:39,896 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1678474522982" table: "testCompactionWithHFileLink" creation_time: 1678474522992 type: FLUSH version: 2 ttl: 0 2023-03-10 18:55:39,897 DEBUG [PEWorker-4] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink 2023-03-10 18:55:39,901 DEBUG [PEWorker-4] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-10 18:55:39,904 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031087b32fcb24684735927cae60c11a58d3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031087b32fcb24684735927cae60c11a58d3 2023-03-10 18:55:39,905 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023031016b7d936a45145d699d46c8a1695d72c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023031016b7d936a45145d699d46c8a1695d72c 2023-03-10 18:55:39,906 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303102963c250214548829d6b7cb4f0ecec34 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303102963c250214548829d6b7cb4f0ecec34 2023-03-10 18:55:39,906 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230310735dacb5b48d4a2faf383d9c602a19e4 of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,910 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303108865f8d67a9843bc90e3c0d40548a2cf of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,913 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230310f0a1cc08ae174323961d9269bdffe719 of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,915 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230310ff85a5ab8138412d8c34bd63cd8dd73f of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,918 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303100110de7bf34f4f06b23c5bb30b8237aa of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,921 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303106a98a00705914156b93966a1ed75b658 of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,923 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230310c4a28d13bb2d42d18e7f25b50a5b7d73 of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,926 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230310f498e39faa23437294854cc72b5586f3 of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,929 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230310094ac61681054c759b8da6cb2a2c5cfa of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,931 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230310676fd2889e2047939798d088407d6846 of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,934 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303107269d9f8e1c844ad81e42a5d32da0a16 of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,936 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230310e4e3b007a2774a7ea06cdc9925ad5eef of snapshot=snaptb-1678474522982 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-10 18:55:39,941 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310355d8c4bf863464e9cefeb20e4255132_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230310355d8c4bf863464e9cefeb20e4255132_del 2023-03-10 18:55:39,942 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230310bcde23db410844c482a3ec3804da69ea_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230310bcde23db410844c482a3ec3804da69ea_del 2023-03-10 18:55:39,943 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303108070b35dcd4b4b53bd45a7e47b2ac200_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303108070b35dcd4b4b53bd45a7e47b2ac200_del 2023-03-10 18:55:39,943 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:39,943 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:39,943 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: e46cb331129008feebe80244602e4706 2023-03-10 18:55:39,947 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a57cddc5664247f2b980ab3e59f716e9_SeqId_29_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a57cddc5664247f2b980ab3e59f716e9_SeqId_29_ 2023-03-10 18:55:39,947 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/8f82ca8420d24d279b297e05e3af3f69 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/8f82ca8420d24d279b297e05e3af3f69 2023-03-10 18:55:39,948 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/7bdfb969c6c74cc596889d4168e818f0_SeqId_29_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/7bdfb969c6c74cc596889d4168e818f0_SeqId_29_ 2023-03-10 18:55:39,949 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a6fa1de5ba504799a7ad832729a2f94d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a6fa1de5ba504799a7ad832729a2f94d 2023-03-10 18:55:39,949 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 751ae9dcd2a84cec82d96a4c3a837ccc of snapshot=snaptb-1678474522982 to region=7278512a19c794d2cb2655c58a8803cf table=testCompactionWithHFileLink 2023-03-10 18:55:39,949 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/1f773ee681ad4010bca9a4d59c208c9f_SeqId_29_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/1f773ee681ad4010bca9a4d59c208c9f_SeqId_29_ 2023-03-10 18:55:39,949 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 72986f3f827c45f884d437495654452c of snapshot=snaptb-1678474522982 to region=b1d206b11bc34c2bde324dcef1ee6a89 table=testCompactionWithHFileLink 2023-03-10 18:55:39,949 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/87ef6b48495d41f582df0e67ae23864d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/87ef6b48495d41f582df0e67ae23864d 2023-03-10 18:55:39,949 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 960884e555924219b3af19c08d09a0fd of snapshot=snaptb-1678474522982 to region=e46cb331129008feebe80244602e4706 table=testCompactionWithHFileLink 2023-03-10 18:55:39,955 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink de7cc23d176440b1a983425b09b04cf9 of snapshot=snaptb-1678474522982 to region=7278512a19c794d2cb2655c58a8803cf table=testCompactionWithHFileLink 2023-03-10 18:55:39,955 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 926b09e4a5d841b4a7926d62a4656bef of snapshot=snaptb-1678474522982 to region=b1d206b11bc34c2bde324dcef1ee6a89 table=testCompactionWithHFileLink 2023-03-10 18:55:39,955 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c264e685deff465a9024c52427e64a02 of snapshot=snaptb-1678474522982 to region=e46cb331129008feebe80244602e4706 table=testCompactionWithHFileLink 2023-03-10 18:55:39,966 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b0ebc27232ba46888a2ac0e59dfff023 of snapshot=snaptb-1678474522982 to region=b1d206b11bc34c2bde324dcef1ee6a89 table=testCompactionWithHFileLink 2023-03-10 18:55:39,966 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 972cf7551eb24a32b6db1f98c0be0339 of snapshot=snaptb-1678474522982 to region=e46cb331129008feebe80244602e4706 table=testCompactionWithHFileLink 2023-03-10 18:55:39,966 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 09edeadf05594761a6182f9e099f17c7 of snapshot=snaptb-1678474522982 to region=7278512a19c794d2cb2655c58a8803cf table=testCompactionWithHFileLink 2023-03-10 18:55:39,969 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0fce4462e53f4f3091ed9d3df87900d2 of snapshot=snaptb-1678474522982 to region=b1d206b11bc34c2bde324dcef1ee6a89 table=testCompactionWithHFileLink 2023-03-10 18:55:39,969 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6c02130ff1b7411db8858605388590dc of snapshot=snaptb-1678474522982 to region=7278512a19c794d2cb2655c58a8803cf table=testCompactionWithHFileLink 2023-03-10 18:55:39,969 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 49b0c444247a463e94f4df031173ec97 of snapshot=snaptb-1678474522982 to region=e46cb331129008feebe80244602e4706 table=testCompactionWithHFileLink 2023-03-10 18:55:39,979 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/2a544eab33ee44328dd4bb97fcd2828b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/2a544eab33ee44328dd4bb97fcd2828b 2023-03-10 18:55:39,979 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ac6f63f68c4f4410b8660234c52ecb68 of snapshot=snaptb-1678474522982 to region=e46cb331129008feebe80244602e4706 table=testCompactionWithHFileLink 2023-03-10 18:55:39,981 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/e317c4aad04b4825b1d7567b55bd59fa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/e317c4aad04b4825b1d7567b55bd59fa 2023-03-10 18:55:39,981 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d67d818c37f74d62a3582f788f5c5e93 of snapshot=snaptb-1678474522982 to region=b1d206b11bc34c2bde324dcef1ee6a89 table=testCompactionWithHFileLink 2023-03-10 18:55:39,981 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/665093e9bec34f3c819fb572256f905a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/665093e9bec34f3c819fb572256f905a 2023-03-10 18:55:39,981 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 42edb717d3864e81adb8b53985567a5d of snapshot=snaptb-1678474522982 to region=7278512a19c794d2cb2655c58a8803cf table=testCompactionWithHFileLink 2023-03-10 18:55:39,989 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 33c96e7f08cf4a1594f5c4700912b827 of snapshot=snaptb-1678474522982 to region=e46cb331129008feebe80244602e4706 table=testCompactionWithHFileLink 2023-03-10 18:55:39,991 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 26f6d99d4af945a7b769fe6ca46296a1 of snapshot=snaptb-1678474522982 to region=7278512a19c794d2cb2655c58a8803cf table=testCompactionWithHFileLink 2023-03-10 18:55:39,991 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0c40ba5afb67448695dd46892c3b5bad of snapshot=snaptb-1678474522982 to region=b1d206b11bc34c2bde324dcef1ee6a89 table=testCompactionWithHFileLink 2023-03-10 18:55:39,992 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 74182c20bf50441cbffc56e7b1fbc79f of snapshot=snaptb-1678474522982 to region=e46cb331129008feebe80244602e4706 table=testCompactionWithHFileLink 2023-03-10 18:55:39,995 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6ab7ac83a5b54c48b8636359b183b0c5 of snapshot=snaptb-1678474522982 to region=7278512a19c794d2cb2655c58a8803cf table=testCompactionWithHFileLink 2023-03-10 18:55:39,995 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f4f5442a622542db857adf9fdfb3f80b of snapshot=snaptb-1678474522982 to region=b1d206b11bc34c2bde324dcef1ee6a89 table=testCompactionWithHFileLink 2023-03-10 18:55:39,996 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9654f0864397433d9743a6560de07ee3 of snapshot=snaptb-1678474522982 to region=e46cb331129008feebe80244602e4706 table=testCompactionWithHFileLink 2023-03-10 18:55:39,999 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink daed0157d94b4114b52074154ee885d3 of snapshot=snaptb-1678474522982 to region=7278512a19c794d2cb2655c58a8803cf table=testCompactionWithHFileLink 2023-03-10 18:55:39,999 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4ced761ac1c0490980f2d865fa31a9e2 of snapshot=snaptb-1678474522982 to region=b1d206b11bc34c2bde324dcef1ee6a89 table=testCompactionWithHFileLink 2023-03-10 18:55:40,002 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1678474522982" table: "testCompactionWithHFileLink" creation_time: 1678474522992 type: FLUSH version: 2 ttl: 0 2023-03-10 18:55:40,003 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678474540003"}]},"ts":"9223372036854775807"} 2023-03-10 18:55:40,003 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678474540003"}]},"ts":"9223372036854775807"} 2023-03-10 18:55:40,003 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678474540003"}]},"ts":"9223372036854775807"} 2023-03-10 18:55:40,005 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-10 18:55:40,005 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7278512a19c794d2cb2655c58a8803cf, NAME => 'testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => b1d206b11bc34c2bde324dcef1ee6a89, NAME => 'testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => e46cb331129008feebe80244602e4706, NAME => 'testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.', STARTKEY => '', ENDKEY => '1'}] 2023-03-10 18:55:40,005 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474540004"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474540004"}]},"ts":"1678474540004"} 2023-03-10 18:55:40,005 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474540004"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474540004"}]},"ts":"1678474540004"} 2023-03-10 18:55:40,005 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474540004"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474540004"}]},"ts":"1678474540004"} 2023-03-10 18:55:40,006 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-10 18:55:40,007 INFO [PEWorker-4] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-10 18:55:40,007 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 7278512a19c794d2cb2655c58a8803cf, NAME => 'testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => b1d206b11bc34c2bde324dcef1ee6a89, NAME => 'testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => e46cb331129008feebe80244602e4706, NAME => 'testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.', STARTKEY => '', ENDKEY => '1'}] 2023-03-10 18:55:40,007 INFO [PEWorker-4] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1678474522982 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-10 18:55:40,009 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678474522982" table: "testCompactionWithHFileLink" creation_time: 1678474522992 type: FLUSH version: 2 ttl: 0 ) in 539 msec 2023-03-10 18:55:40,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-10 18:55:40,086 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-10 18:55:40,086 INFO [Listener at localhost/36413] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1678474522982-1678474537314 2023-03-10 18:55:40,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1678474522982-1678474537314" 2023-03-10 18:55:40,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1678474522982-1678474537314 2023-03-10 18:55:40,095 INFO [Listener at localhost/36413] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-10 18:55:40,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-03-10 18:55:40,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-10 18:55:40,109 INFO [PEWorker-5] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-10 18:55:40,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-10 18:55:40,109 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474540109"}]},"ts":"1678474540109"} 2023-03-10 18:55:40,110 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-10 18:55:40,112 DEBUG [PEWorker-5] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-10 18:55:40,113 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e46cb331129008feebe80244602e4706, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b1d206b11bc34c2bde324dcef1ee6a89, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7278512a19c794d2cb2655c58a8803cf, ASSIGN}] 2023-03-10 18:55:40,114 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b1d206b11bc34c2bde324dcef1ee6a89, ASSIGN 2023-03-10 18:55:40,114 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7278512a19c794d2cb2655c58a8803cf, ASSIGN 2023-03-10 18:55:40,114 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e46cb331129008feebe80244602e4706, ASSIGN 2023-03-10 18:55:40,115 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7278512a19c794d2cb2655c58a8803cf, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-10 18:55:40,115 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e46cb331129008feebe80244602e4706, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-10 18:55:40,115 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b1d206b11bc34c2bde324dcef1ee6a89, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-10 18:55:40,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-10 18:55:40,267 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=e46cb331129008feebe80244602e4706, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:40,267 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=b1d206b11bc34c2bde324dcef1ee6a89, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:40,267 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=7278512a19c794d2cb2655c58a8803cf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:40,267 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474540267"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474540267"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474540267"}]},"ts":"1678474540267"} 2023-03-10 18:55:40,267 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474540267"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474540267"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474540267"}]},"ts":"1678474540267"} 2023-03-10 18:55:40,267 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474540267"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474540267"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474540267"}]},"ts":"1678474540267"} 2023-03-10 18:55:40,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=98, state=RUNNABLE; OpenRegionProcedure e46cb331129008feebe80244602e4706, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:40,270 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=100, state=RUNNABLE; OpenRegionProcedure 7278512a19c794d2cb2655c58a8803cf, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:40,271 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure b1d206b11bc34c2bde324dcef1ee6a89, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:40,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-10 18:55:40,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:40,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7278512a19c794d2cb2655c58a8803cf, NAME => 'testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.', STARTKEY => '2', ENDKEY => ''} 2023-03-10 18:55:40,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:40,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:40,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:40,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:40,425 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:40,425 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1 2023-03-10 18:55:40,426 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1 2023-03-10 18:55:40,426 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7278512a19c794d2cb2655c58a8803cf columnFamilyName family1 2023-03-10 18:55:40,441 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-09edeadf05594761a6182f9e099f17c7 2023-03-10 18:55:40,447 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6c02130ff1b7411db8858605388590dc 2023-03-10 18:55:40,452 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-751ae9dcd2a84cec82d96a4c3a837ccc 2023-03-10 18:55:40,456 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-de7cc23d176440b1a983425b09b04cf9 2023-03-10 18:55:40,457 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(310): Store=7278512a19c794d2cb2655c58a8803cf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:40,457 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:40,458 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2 2023-03-10 18:55:40,458 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2 2023-03-10 18:55:40,458 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7278512a19c794d2cb2655c58a8803cf columnFamilyName family2 2023-03-10 18:55:40,465 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-26f6d99d4af945a7b769fe6ca46296a1 2023-03-10 18:55:40,470 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-42edb717d3864e81adb8b53985567a5d 2023-03-10 18:55:40,476 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6ab7ac83a5b54c48b8636359b183b0c5 2023-03-10 18:55:40,482 DEBUG [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-daed0157d94b4114b52074154ee885d3 2023-03-10 18:55:40,482 INFO [StoreOpener-7278512a19c794d2cb2655c58a8803cf-1] regionserver.HStore(310): Store=7278512a19c794d2cb2655c58a8803cf/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:40,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:40,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:40,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:55:40,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:55:40,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7278512a19c794d2cb2655c58a8803cf; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71323186, jitterRate=0.06279829144477844}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:55:40,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:40,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf., pid=102, masterSystemTime=1678474540420 2023-03-10 18:55:40,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:55:40,491 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-10 18:55:40,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-10 18:55:40,491 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-10 18:55:40,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:40,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:40,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:40,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b1d206b11bc34c2bde324dcef1ee6a89, NAME => 'testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.', STARTKEY => '1', ENDKEY => '2'} 2023-03-10 18:55:40,493 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=7278512a19c794d2cb2655c58a8803cf, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:40,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:40,493 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474540493"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474540493"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474540493"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474540493"}]},"ts":"1678474540493"} 2023-03-10 18:55:40,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:40,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:40,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:40,496 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-10 18:55:40,496 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HStore(1912): 7278512a19c794d2cb2655c58a8803cf/family2 is initiating minor compaction (all files) 2023-03-10 18:55:40,496 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 7278512a19c794d2cb2655c58a8803cf/family2 in testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:40,496 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-42edb717d3864e81adb8b53985567a5d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-26f6d99d4af945a7b769fe6ca46296a1, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6ab7ac83a5b54c48b8636359b183b0c5, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-daed0157d94b4114b52074154ee885d3] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp, totalSize=20.4 K 2023-03-10 18:55:40,496 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=100 2023-03-10 18:55:40,497 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=100, state=SUCCESS; OpenRegionProcedure 7278512a19c794d2cb2655c58a8803cf, server=jenkins-hbase4.apache.org,43181,1678474414959 in 225 msec 2023-03-10 18:55:40,497 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:40,497 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-10 18:55:40,497 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1912): 7278512a19c794d2cb2655c58a8803cf/family1 is initiating minor compaction (all files) 2023-03-10 18:55:40,497 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7278512a19c794d2cb2655c58a8803cf/family1 in testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:55:40,498 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-751ae9dcd2a84cec82d96a4c3a837ccc, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-de7cc23d176440b1a983425b09b04cf9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-09edeadf05594761a6182f9e099f17c7, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6c02130ff1b7411db8858605388590dc] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp, totalSize=21.6 K 2023-03-10 18:55:40,498 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7278512a19c794d2cb2655c58a8803cf, ASSIGN in 384 msec 2023-03-10 18:55:40,498 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-42edb717d3864e81adb8b53985567a5d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515817 2023-03-10 18:55:40,499 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-751ae9dcd2a84cec82d96a4c3a837ccc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515817 2023-03-10 18:55:40,499 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1 2023-03-10 18:55:40,499 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1 2023-03-10 18:55:40,499 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-26f6d99d4af945a7b769fe6ca46296a1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517607 2023-03-10 18:55:40,499 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-de7cc23d176440b1a983425b09b04cf9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517607 2023-03-10 18:55:40,500 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b1d206b11bc34c2bde324dcef1ee6a89 columnFamilyName family1 2023-03-10 18:55:40,501 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6ab7ac83a5b54c48b8636359b183b0c5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:40,501 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-09edeadf05594761a6182f9e099f17c7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:40,502 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-daed0157d94b4114b52074154ee885d3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521193 2023-03-10 18:55:40,502 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6c02130ff1b7411db8858605388590dc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521193 2023-03-10 18:55:40,525 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0fce4462e53f4f3091ed9d3df87900d2 2023-03-10 18:55:40,536 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-72986f3f827c45f884d437495654452c 2023-03-10 18:55:40,549 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-926b09e4a5d841b4a7926d62a4656bef 2023-03-10 18:55:40,561 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7278512a19c794d2cb2655c58a8803cf#family1#compaction#113 average throughput is 0.18 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:40,561 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] throttle.PressureAwareThroughputController(145): 7278512a19c794d2cb2655c58a8803cf#family2#compaction#114 average throughput is 0.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:40,569 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-b0ebc27232ba46888a2ac0e59dfff023 2023-03-10 18:55:40,572 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(310): Store=b1d206b11bc34c2bde324dcef1ee6a89/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:40,572 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:40,573 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2 2023-03-10 18:55:40,573 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2 2023-03-10 18:55:40,574 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b1d206b11bc34c2bde324dcef1ee6a89 columnFamilyName family2 2023-03-10 18:55:40,585 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0c40ba5afb67448695dd46892c3b5bad 2023-03-10 18:55:40,589 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-4ced761ac1c0490980f2d865fa31a9e2 2023-03-10 18:55:40,594 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-d67d818c37f74d62a3582f788f5c5e93 2023-03-10 18:55:40,597 DEBUG [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-f4f5442a622542db857adf9fdfb3f80b 2023-03-10 18:55:40,598 INFO [StoreOpener-b1d206b11bc34c2bde324dcef1ee6a89-1] regionserver.HStore(310): Store=b1d206b11bc34c2bde324dcef1ee6a89/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:40,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:40,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:40,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:55:40,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:55:40,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b1d206b11bc34c2bde324dcef1ee6a89; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68074921, jitterRate=0.014395371079444885}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:55:40,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:40,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89., pid=103, masterSystemTime=1678474540420 2023-03-10 18:55:40,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-10 18:55:40,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-10 18:55:40,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:40,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:40,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:40,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e46cb331129008feebe80244602e4706, NAME => 'testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.', STARTKEY => '', ENDKEY => '1'} 2023-03-10 18:55:40,605 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=b1d206b11bc34c2bde324dcef1ee6a89, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:40,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink e46cb331129008feebe80244602e4706 2023-03-10 18:55:40,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:40,605 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474540605"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474540605"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474540605"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474540605"}]},"ts":"1678474540605"} 2023-03-10 18:55:40,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e46cb331129008feebe80244602e4706 2023-03-10 18:55:40,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e46cb331129008feebe80244602e4706 2023-03-10 18:55:40,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=99 2023-03-10 18:55:40,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure b1d206b11bc34c2bde324dcef1ee6a89, server=jenkins-hbase4.apache.org,43181,1678474414959 in 335 msec 2023-03-10 18:55:40,608 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b1d206b11bc34c2bde324dcef1ee6a89, ASSIGN in 494 msec 2023-03-10 18:55:40,611 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e46cb331129008feebe80244602e4706 2023-03-10 18:55:40,612 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1 2023-03-10 18:55:40,612 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1 2023-03-10 18:55:40,613 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e46cb331129008feebe80244602e4706 columnFamilyName family1 2023-03-10 18:55:40,618 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-49b0c444247a463e94f4df031173ec97 2023-03-10 18:55:40,623 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-960884e555924219b3af19c08d09a0fd 2023-03-10 18:55:40,627 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-972cf7551eb24a32b6db1f98c0be0339 2023-03-10 18:55:40,631 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-c264e685deff465a9024c52427e64a02 2023-03-10 18:55:40,631 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(310): Store=e46cb331129008feebe80244602e4706/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:40,632 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region e46cb331129008feebe80244602e4706 2023-03-10 18:55:40,632 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2 2023-03-10 18:55:40,632 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2 2023-03-10 18:55:40,633 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e46cb331129008feebe80244602e4706 columnFamilyName family2 2023-03-10 18:55:40,638 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-33c96e7f08cf4a1594f5c4700912b827 2023-03-10 18:55:40,641 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-74182c20bf50441cbffc56e7b1fbc79f 2023-03-10 18:55:40,646 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-9654f0864397433d9743a6560de07ee3 2023-03-10 18:55:40,651 DEBUG [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(539): loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-ac6f63f68c4f4410b8660234c52ecb68 2023-03-10 18:55:40,651 INFO [StoreOpener-e46cb331129008feebe80244602e4706-1] regionserver.HStore(310): Store=e46cb331129008feebe80244602e4706/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:40,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706 2023-03-10 18:55:40,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706 2023-03-10 18:55:40,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:55:40,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e46cb331129008feebe80244602e4706 2023-03-10 18:55:40,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e46cb331129008feebe80244602e4706; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65292862, jitterRate=-0.02706053853034973}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:55:40,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:40,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706., pid=101, masterSystemTime=1678474540420 2023-03-10 18:55:40,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-10 18:55:40,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-10 18:55:40,657 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:40,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:40,658 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=e46cb331129008feebe80244602e4706, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:40,658 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678474540657"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474540657"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474540657"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474540657"}]},"ts":"1678474540657"} 2023-03-10 18:55:40,661 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=98 2023-03-10 18:55:40,661 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=98, state=SUCCESS; OpenRegionProcedure e46cb331129008feebe80244602e4706, server=jenkins-hbase4.apache.org,43181,1678474414959 in 390 msec 2023-03-10 18:55:40,662 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=98, resume processing ppid=97 2023-03-10 18:55:40,662 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=e46cb331129008feebe80244602e4706, ASSIGN in 548 msec 2023-03-10 18:55:40,663 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474540663"}]},"ts":"1678474540663"} 2023-03-10 18:55:40,664 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-10 18:55:40,666 INFO [PEWorker-2] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-10 18:55:40,667 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 565 msec 2023-03-10 18:55:40,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-10 18:55:40,712 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-10 18:55:40,724 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:40,730 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:40,737 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:40,738 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:40,743 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-10 18:55:40,755 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-10 18:55:40,756 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:40,756 DEBUG [Listener at localhost/36413] zookeeper.ReadOnlyZKClient(139): Connect 0x236a9481 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:55:40,762 DEBUG [Listener at localhost/36413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37a80859, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:55:40,762 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:40,762 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:40,762 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:41,388 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family2/7bc8a3568e544cf783b3ff6f09648fff as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/7bc8a3568e544cf783b3ff6f09648fff 2023-03-10 18:55:41,392 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 7278512a19c794d2cb2655c58a8803cf/family2 of 7278512a19c794d2cb2655c58a8803cf into 7bc8a3568e544cf783b3ff6f09648fff(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:55:41,392 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:41,392 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf., storeName=7278512a19c794d2cb2655c58a8803cf/family2, priority=12, startTime=1678474540491; duration=0sec 2023-03-10 18:55:41,393 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-10 18:55:41,393 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-10 18:55:41,395 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-10 18:55:41,395 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HStore(1912): b1d206b11bc34c2bde324dcef1ee6a89/family1 is initiating minor compaction (all files) 2023-03-10 18:55:41,396 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HRegion(2259): Starting compaction of b1d206b11bc34c2bde324dcef1ee6a89/family1 in testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:41,396 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-72986f3f827c45f884d437495654452c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-926b09e4a5d841b4a7926d62a4656bef, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-b0ebc27232ba46888a2ac0e59dfff023, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0fce4462e53f4f3091ed9d3df87900d2] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp, totalSize=21.6 K 2023-03-10 18:55:41,396 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-72986f3f827c45f884d437495654452c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515816 2023-03-10 18:55:41,397 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-926b09e4a5d841b4a7926d62a4656bef, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517606 2023-03-10 18:55:41,397 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-b0ebc27232ba46888a2ac0e59dfff023, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:41,398 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0fce4462e53f4f3091ed9d3df87900d2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521192 2023-03-10 18:55:41,409 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] throttle.PressureAwareThroughputController(145): b1d206b11bc34c2bde324dcef1ee6a89#family1#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:41,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-42edb717d3864e81adb8b53985567a5d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-26f6d99d4af945a7b769fe6ca46296a1, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6ab7ac83a5b54c48b8636359b183b0c5, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-daed0157d94b4114b52074154ee885d3] to archive 2023-03-10 18:55:41,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:41,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-42edb717d3864e81adb8b53985567a5d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-42edb717d3864e81adb8b53985567a5d 2023-03-10 18:55:41,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-26f6d99d4af945a7b769fe6ca46296a1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-26f6d99d4af945a7b769fe6ca46296a1 2023-03-10 18:55:41,414 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6ab7ac83a5b54c48b8636359b183b0c5 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6ab7ac83a5b54c48b8636359b183b0c5 2023-03-10 18:55:41,415 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-daed0157d94b4114b52074154ee885d3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family2/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-daed0157d94b4114b52074154ee885d3 2023-03-10 18:55:41,609 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303109c7f8173de5243919af065a2a5758c28 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303109c7f8173de5243919af065a2a5758c28 2023-03-10 18:55:41,610 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:41,614 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310adc8324c4f1d49a08dd873a9e44e584b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230310adc8324c4f1d49a08dd873a9e44e584b 2023-03-10 18:55:41,615 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:41,619 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303101512afbf4c6640d38a430e3df0c5312a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303101512afbf4c6640d38a430e3df0c5312a 2023-03-10 18:55:41,619 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:41,625 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:55:41,629 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41748, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:55:41,638 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:55:41,647 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34244, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:55:41,675 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230310/family1/78d3d86688d94c22bbad2b270dadd82f first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-10 18:55:41,676 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=32 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230310/family1/78d3d86688d94c22bbad2b270dadd82f}] 2023-03-10 18:55:41,676 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230310/family1/5e92a572af8e4450ad3f11c5a778eb4f first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-10 18:55:41,678 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230310/family1/ad9dac16bc62419bb92eae34ca7492ba first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-10 18:55:41,679 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230310/family1/5e92a572af8e4450ad3f11c5a778eb4f}] 2023-03-10 18:55:41,680 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230310/family1/ad9dac16bc62419bb92eae34ca7492ba}] 2023-03-10 18:55:41,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230310/family1/ad9dac16bc62419bb92eae34ca7492ba for inclusion in b1d206b11bc34c2bde324dcef1ee6a89/family1 2023-03-10 18:55:41,690 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230310/family1/78d3d86688d94c22bbad2b270dadd82f for inclusion in e46cb331129008feebe80244602e4706/family1 2023-03-10 18:55:41,690 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230310/family1/5e92a572af8e4450ad3f11c5a778eb4f for inclusion in 7278512a19c794d2cb2655c58a8803cf/family1 2023-03-10 18:55:41,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-10 18:55:41,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-10 18:55:41,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-10 18:55:41,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-10 18:55:41,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:41,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43181] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-10 18:55:41,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last=1 2023-03-10 18:55:41,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:41,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:41,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230310/family1/5e92a572af8e4450ad3f11c5a778eb4f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__pahj84jcotrfktkd7jhrq45q8ae644cb7cdmo0rl0tljst6hqh7v70uro92v2skt/family1/5e92a572af8e4450ad3f11c5a778eb4f 2023-03-10 18:55:41,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230310/family1/ad9dac16bc62419bb92eae34ca7492ba to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__mh3ak1hass7e1nhp648eavi29ti2b9u1h3fokt6qug9suli5pbg18pmen3cakjr2/family1/ad9dac16bc62419bb92eae34ca7492ba 2023-03-10 18:55:41,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230310/family1/78d3d86688d94c22bbad2b270dadd82f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__tf02f8ksolulntq9idba8k3v110min00lkg20n48a6i54drfp3lqihun54ec6nhe/family1/78d3d86688d94c22bbad2b270dadd82f 2023-03-10 18:55:41,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__mh3ak1hass7e1nhp648eavi29ti2b9u1h3fokt6qug9suli5pbg18pmen3cakjr2/family1/ad9dac16bc62419bb92eae34ca7492ba as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/081e183e943e4275a19d492f3f61d33e_SeqId_34_ 2023-03-10 18:55:41,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__mh3ak1hass7e1nhp648eavi29ti2b9u1h3fokt6qug9suli5pbg18pmen3cakjr2/family1/ad9dac16bc62419bb92eae34ca7492ba into b1d206b11bc34c2bde324dcef1ee6a89/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/081e183e943e4275a19d492f3f61d33e_SeqId_34_ - updating store file list. 2023-03-10 18:55:41,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__pahj84jcotrfktkd7jhrq45q8ae644cb7cdmo0rl0tljst6hqh7v70uro92v2skt/family1/5e92a572af8e4450ad3f11c5a778eb4f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/c35ea29abd7446468449376ed5ef42be_SeqId_35_ 2023-03-10 18:55:41,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__tf02f8ksolulntq9idba8k3v110min00lkg20n48a6i54drfp3lqihun54ec6nhe/family1/78d3d86688d94c22bbad2b270dadd82f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/bbd92dea2ef64bdaa5064782ca107d0b_SeqId_34_ 2023-03-10 18:55:41,701 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__pahj84jcotrfktkd7jhrq45q8ae644cb7cdmo0rl0tljst6hqh7v70uro92v2skt/family1/5e92a572af8e4450ad3f11c5a778eb4f into 7278512a19c794d2cb2655c58a8803cf/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/c35ea29abd7446468449376ed5ef42be_SeqId_35_ - updating store file list. 2023-03-10 18:55:41,702 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__tf02f8ksolulntq9idba8k3v110min00lkg20n48a6i54drfp3lqihun54ec6nhe/family1/78d3d86688d94c22bbad2b270dadd82f into e46cb331129008feebe80244602e4706/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/bbd92dea2ef64bdaa5064782ca107d0b_SeqId_34_ - updating store file list. 2023-03-10 18:55:41,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/081e183e943e4275a19d492f3f61d33e_SeqId_34_ into b1d206b11bc34c2bde324dcef1ee6a89/family1 2023-03-10 18:55:41,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__mh3ak1hass7e1nhp648eavi29ti2b9u1h3fokt6qug9suli5pbg18pmen3cakjr2/family1/ad9dac16bc62419bb92eae34ca7492ba into b1d206b11bc34c2bde324dcef1ee6a89/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/081e183e943e4275a19d492f3f61d33e_SeqId_34_) 2023-03-10 18:55:41,706 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/c35ea29abd7446468449376ed5ef42be_SeqId_35_ into 7278512a19c794d2cb2655c58a8803cf/family1 2023-03-10 18:55:41,706 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__pahj84jcotrfktkd7jhrq45q8ae644cb7cdmo0rl0tljst6hqh7v70uro92v2skt/family1/5e92a572af8e4450ad3f11c5a778eb4f into 7278512a19c794d2cb2655c58a8803cf/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/c35ea29abd7446468449376ed5ef42be_SeqId_35_) 2023-03-10 18:55:41,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__mh3ak1hass7e1nhp648eavi29ti2b9u1h3fokt6qug9suli5pbg18pmen3cakjr2/family1/ad9dac16bc62419bb92eae34ca7492ba 2023-03-10 18:55:41,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__pahj84jcotrfktkd7jhrq45q8ae644cb7cdmo0rl0tljst6hqh7v70uro92v2skt/family1/5e92a572af8e4450ad3f11c5a778eb4f 2023-03-10 18:55:41,709 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/bbd92dea2ef64bdaa5064782ca107d0b_SeqId_34_ into e46cb331129008feebe80244602e4706/family1 2023-03-10 18:55:41,709 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__tf02f8ksolulntq9idba8k3v110min00lkg20n48a6i54drfp3lqihun54ec6nhe/family1/78d3d86688d94c22bbad2b270dadd82f into e46cb331129008feebe80244602e4706/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/bbd92dea2ef64bdaa5064782ca107d0b_SeqId_34_) 2023-03-10 18:55:41,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testCompactionWithHFileLink__tf02f8ksolulntq9idba8k3v110min00lkg20n48a6i54drfp3lqihun54ec6nhe/family1/78d3d86688d94c22bbad2b270dadd82f 2023-03-10 18:55:41,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:41,710 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:41,710 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:41,710 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:41,711 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:41,713 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230310735dacb5b48d4a2faf383d9c602a19e4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230310735dacb5b48d4a2faf383d9c602a19e4 2023-03-10 18:55:41,713 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303100110de7bf34f4f06b23c5bb30b8237aa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303100110de7bf34f4f06b23c5bb30b8237aa 2023-03-10 18:55:41,714 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303106a98a00705914156b93966a1ed75b658 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303106a98a00705914156b93966a1ed75b658 2023-03-10 18:55:41,715 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303108865f8d67a9843bc90e3c0d40548a2cf to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303108865f8d67a9843bc90e3c0d40548a2cf 2023-03-10 18:55:41,716 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230310c4a28d13bb2d42d18e7f25b50a5b7d73 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230310c4a28d13bb2d42d18e7f25b50a5b7d73 2023-03-10 18:55:41,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-10 18:55:41,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(1912): e46cb331129008feebe80244602e4706/family1 is initiating minor compaction (all files) 2023-03-10 18:55:41,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2b53e5d4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-10 18:55:41,717 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230310f0a1cc08ae174323961d9269bdffe719 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230310f0a1cc08ae174323961d9269bdffe719 2023-03-10 18:55:41,718 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230310f498e39faa23437294854cc72b5586f3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230310f498e39faa23437294854cc72b5586f3 2023-03-10 18:55:41,718 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-10 18:55:41,718 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230310' 2023-03-10 18:55:41,718 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230310ff85a5ab8138412d8c34bd63cd8dd73f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230310ff85a5ab8138412d8c34bd63cd8dd73f 2023-03-10 18:55:41,718 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-10 18:55:41,718 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230310' 2023-03-10 18:55:41,719 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:41,719 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:41,719 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230310 table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:41,720 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:41,721 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230310094ac61681054c759b8da6cb2a2c5cfa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230310094ac61681054c759b8da6cb2a2c5cfa 2023-03-10 18:55:41,722 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230310676fd2889e2047939798d088407d6846 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230310676fd2889e2047939798d088407d6846 2023-03-10 18:55:41,723 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303107269d9f8e1c844ad81e42a5d32da0a16 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303107269d9f8e1c844ad81e42a5d32da0a16 2023-03-10 18:55:41,724 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230310e4e3b007a2774a7ea06cdc9925ad5eef to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230310e4e3b007a2774a7ea06cdc9925ad5eef 2023-03-10 18:55:41,724 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-10 18:55:41,724 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230310' 2023-03-10 18:55:41,725 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230310 for table='testCompactionWithHFileLink' and column='family1' 2023-03-10 18:55:41,725 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-10 18:55:41,728 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:41,731 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:41,736 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:41,737 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:41,742 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:41,743 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:41,764 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=284 (was 236) - Thread LEAK? -, OpenFileDescriptor=758 (was 598) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=189 (was 202), ProcessCount=170 (was 170), AvailableMemoryMB=5431 (was 5530) 2023-03-10 18:55:41,778 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=284, OpenFileDescriptor=758, MaxFileDescriptor=60000, SystemLoadAverage=189, ProcessCount=170, AvailableMemoryMB=5431 2023-03-10 18:55:41,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-03-10 18:55:41,784 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/.tmp/family1/a85da777d7364e2abdc669139ac956e7 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/a85da777d7364e2abdc669139ac956e7 2023-03-10 18:55:41,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-10 18:55:41,790 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 7278512a19c794d2cb2655c58a8803cf/family1 of 7278512a19c794d2cb2655c58a8803cf into a85da777d7364e2abdc669139ac956e7(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:41,790 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:55:41,790 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf., storeName=7278512a19c794d2cb2655c58a8803cf/family1, priority=12, startTime=1678474540490; duration=1sec 2023-03-10 18:55:41,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-10 18:55:41,790 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-10 18:55:41,790 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e46cb331129008feebe80244602e4706/family1 in testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:41,790 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-960884e555924219b3af19c08d09a0fd, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-c264e685deff465a9024c52427e64a02, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-972cf7551eb24a32b6db1f98c0be0339, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-49b0c444247a463e94f4df031173ec97, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/bbd92dea2ef64bdaa5064782ca107d0b_SeqId_34_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp, totalSize=28.9 K 2023-03-10 18:55:41,791 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-960884e555924219b3af19c08d09a0fd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515815 2023-03-10 18:55:41,791 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-10 18:55:41,792 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-c264e685deff465a9024c52427e64a02, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517606 2023-03-10 18:55:41,792 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-972cf7551eb24a32b6db1f98c0be0339, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:41,793 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-49b0c444247a463e94f4df031173ec97, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521192 2023-03-10 18:55:41,794 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting bbd92dea2ef64bdaa5064782ca107d0b_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678474515815 2023-03-10 18:55:41,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 8 msec 2023-03-10 18:55:41,808 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): e46cb331129008feebe80244602e4706#family1#compaction#116 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:41,810 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-751ae9dcd2a84cec82d96a4c3a837ccc, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-de7cc23d176440b1a983425b09b04cf9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-09edeadf05594761a6182f9e099f17c7, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6c02130ff1b7411db8858605388590dc] to archive 2023-03-10 18:55:41,811 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:41,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-751ae9dcd2a84cec82d96a4c3a837ccc to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-751ae9dcd2a84cec82d96a4c3a837ccc 2023-03-10 18:55:41,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-de7cc23d176440b1a983425b09b04cf9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-de7cc23d176440b1a983425b09b04cf9 2023-03-10 18:55:41,817 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-09edeadf05594761a6182f9e099f17c7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-09edeadf05594761a6182f9e099f17c7 2023-03-10 18:55:41,818 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6c02130ff1b7411db8858605388590dc to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/family1/testCompactionWithHFileLink=7278512a19c794d2cb2655c58a8803cf-6c02130ff1b7411db8858605388590dc 2023-03-10 18:55:41,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-10 18:55:41,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-10 18:55:41,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-10 18:55:41,895 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:55:41,895 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-10 18:55:41,895 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:55:41,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-10 18:55:41,897 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227 2023-03-10 18:55:41,897 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633 2023-03-10 18:55:41,897 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774 2023-03-10 18:55:41,897 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227 empty. 2023-03-10 18:55:41,898 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633 empty. 2023-03-10 18:55:41,898 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774 empty. 2023-03-10 18:55:41,898 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227 2023-03-10 18:55:41,898 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774 2023-03-10 18:55:41,898 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633 2023-03-10 18:55:41,898 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-10 18:55:41,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-10 18:55:42,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-10 18:55:42,308 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-10 18:55:42,310 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 86d4ba0f8509f65063fb59feedda6774, NAME => 'ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:55:42,310 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6cce5e62535147bca4bf736faaf0f227, NAME => 'ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:55:42,310 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => 3e4f64f1240b6c6abb2116376732f633, NAME => 'ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:55:42,318 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:42,318 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing 6cce5e62535147bca4bf736faaf0f227, disabling compactions & flushes 2023-03-10 18:55:42,318 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:42,318 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:42,318 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. after waiting 0 ms 2023-03-10 18:55:42,318 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:42,318 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:42,318 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 6cce5e62535147bca4bf736faaf0f227: 2023-03-10 18:55:42,319 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:42,319 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 86d4ba0f8509f65063fb59feedda6774, disabling compactions & flushes 2023-03-10 18:55:42,319 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:42,319 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:42,319 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. after waiting 0 ms 2023-03-10 18:55:42,319 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:42,319 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:42,319 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 86d4ba0f8509f65063fb59feedda6774: 2023-03-10 18:55:42,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-10 18:55:42,637 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family1/0ea35e72722647dc9915878c24d7cf14 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/0ea35e72722647dc9915878c24d7cf14 2023-03-10 18:55:42,642 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in b1d206b11bc34c2bde324dcef1ee6a89/family1 of b1d206b11bc34c2bde324dcef1ee6a89 into 0ea35e72722647dc9915878c24d7cf14(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:42,642 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:42,642 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89., storeName=b1d206b11bc34c2bde324dcef1ee6a89/family1, priority=12, startTime=1678474540603; duration=1sec 2023-03-10 18:55:42,642 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-10 18:55:42,642 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-10 18:55:42,644 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-10 18:55:42,644 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HStore(1912): b1d206b11bc34c2bde324dcef1ee6a89/family2 is initiating minor compaction (all files) 2023-03-10 18:55:42,644 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HRegion(2259): Starting compaction of b1d206b11bc34c2bde324dcef1ee6a89/family2 in testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:55:42,644 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-d67d818c37f74d62a3582f788f5c5e93, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0c40ba5afb67448695dd46892c3b5bad, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-f4f5442a622542db857adf9fdfb3f80b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-4ced761ac1c0490980f2d865fa31a9e2] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp, totalSize=20.4 K 2023-03-10 18:55:42,645 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-d67d818c37f74d62a3582f788f5c5e93, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515816 2023-03-10 18:55:42,645 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0c40ba5afb67448695dd46892c3b5bad, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517606 2023-03-10 18:55:42,646 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-f4f5442a622542db857adf9fdfb3f80b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:42,646 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-4ced761ac1c0490980f2d865fa31a9e2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521192 2023-03-10 18:55:42,657 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] throttle.PressureAwareThroughputController(145): b1d206b11bc34c2bde324dcef1ee6a89#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:42,699 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:55:42,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-72986f3f827c45f884d437495654452c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-926b09e4a5d841b4a7926d62a4656bef, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-b0ebc27232ba46888a2ac0e59dfff023, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0fce4462e53f4f3091ed9d3df87900d2] to archive 2023-03-10 18:55:42,719 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:42,719 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:42,719 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing 3e4f64f1240b6c6abb2116376732f633, disabling compactions & flushes 2023-03-10 18:55:42,719 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:42,719 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:42,719 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. after waiting 0 ms 2023-03-10 18:55:42,719 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:42,719 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:42,719 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for 3e4f64f1240b6c6abb2116376732f633: 2023-03-10 18:55:42,722 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-72986f3f827c45f884d437495654452c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-72986f3f827c45f884d437495654452c 2023-03-10 18:55:42,722 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:55:42,723 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678474542723"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474542723"}]},"ts":"1678474542723"} 2023-03-10 18:55:42,723 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678474542723"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474542723"}]},"ts":"1678474542723"} 2023-03-10 18:55:42,723 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678474542723"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474542723"}]},"ts":"1678474542723"} 2023-03-10 18:55:42,724 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-926b09e4a5d841b4a7926d62a4656bef to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-926b09e4a5d841b4a7926d62a4656bef 2023-03-10 18:55:42,725 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-10 18:55:42,726 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:55:42,726 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474542726"}]},"ts":"1678474542726"} 2023-03-10 18:55:42,727 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-b0ebc27232ba46888a2ac0e59dfff023 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-b0ebc27232ba46888a2ac0e59dfff023 2023-03-10 18:55:42,728 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-10 18:55:42,729 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0fce4462e53f4f3091ed9d3df87900d2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family1/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0fce4462e53f4f3091ed9d3df87900d2 2023-03-10 18:55:42,732 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6cce5e62535147bca4bf736faaf0f227, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3e4f64f1240b6c6abb2116376732f633, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=86d4ba0f8509f65063fb59feedda6774, ASSIGN}] 2023-03-10 18:55:42,734 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6cce5e62535147bca4bf736faaf0f227, ASSIGN 2023-03-10 18:55:42,734 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=86d4ba0f8509f65063fb59feedda6774, ASSIGN 2023-03-10 18:55:42,735 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3e4f64f1240b6c6abb2116376732f633, ASSIGN 2023-03-10 18:55:42,735 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=86d4ba0f8509f65063fb59feedda6774, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:55:42,735 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3e4f64f1240b6c6abb2116376732f633, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:55:42,735 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6cce5e62535147bca4bf736faaf0f227, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:55:42,889 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=6cce5e62535147bca4bf736faaf0f227, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:42,889 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=86d4ba0f8509f65063fb59feedda6774, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:42,889 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=3e4f64f1240b6c6abb2116376732f633, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:42,889 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678474542889"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474542889"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474542889"}]},"ts":"1678474542889"} 2023-03-10 18:55:42,889 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678474542889"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474542889"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474542889"}]},"ts":"1678474542889"} 2023-03-10 18:55:42,889 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678474542889"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474542889"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474542889"}]},"ts":"1678474542889"} 2023-03-10 18:55:42,890 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=106, state=RUNNABLE; OpenRegionProcedure 6cce5e62535147bca4bf736faaf0f227, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:42,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=107, state=RUNNABLE; OpenRegionProcedure 3e4f64f1240b6c6abb2116376732f633, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:42,891 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure 86d4ba0f8509f65063fb59feedda6774, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:55:43,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-10 18:55:43,035 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family1/b9de242d540c4935a7df5ac1336661ed as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/b9de242d540c4935a7df5ac1336661ed 2023-03-10 18:55:43,039 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in e46cb331129008feebe80244602e4706/family1 of e46cb331129008feebe80244602e4706 into b9de242d540c4935a7df5ac1336661ed(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:43,039 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:43,039 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678474541716; duration=1sec 2023-03-10 18:55:43,039 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-10 18:55:43,039 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-10 18:55:43,041 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-10 18:55:43,041 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1912): e46cb331129008feebe80244602e4706/family2 is initiating minor compaction (all files) 2023-03-10 18:55:43,041 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e46cb331129008feebe80244602e4706/family2 in testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:55:43,041 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-ac6f63f68c4f4410b8660234c52ecb68, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-33c96e7f08cf4a1594f5c4700912b827, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-74182c20bf50441cbffc56e7b1fbc79f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-9654f0864397433d9743a6560de07ee3] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp, totalSize=20.4 K 2023-03-10 18:55:43,042 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-ac6f63f68c4f4410b8660234c52ecb68, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474515815 2023-03-10 18:55:43,043 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-33c96e7f08cf4a1594f5c4700912b827, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474517606 2023-03-10 18:55:43,043 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-74182c20bf50441cbffc56e7b1fbc79f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474519409 2023-03-10 18:55:43,044 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-9654f0864397433d9743a6560de07ee3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474521192 2023-03-10 18:55:43,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:43,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6cce5e62535147bca4bf736faaf0f227, NAME => 'ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227.', STARTKEY => '', ENDKEY => '1'} 2023-03-10 18:55:43,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 6cce5e62535147bca4bf736faaf0f227 2023-03-10 18:55:43,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:43,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6cce5e62535147bca4bf736faaf0f227 2023-03-10 18:55:43,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6cce5e62535147bca4bf736faaf0f227 2023-03-10 18:55:43,046 INFO [StoreOpener-6cce5e62535147bca4bf736faaf0f227-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6cce5e62535147bca4bf736faaf0f227 2023-03-10 18:55:43,047 DEBUG [StoreOpener-6cce5e62535147bca4bf736faaf0f227-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1 2023-03-10 18:55:43,047 DEBUG [StoreOpener-6cce5e62535147bca4bf736faaf0f227-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1 2023-03-10 18:55:43,048 INFO [StoreOpener-6cce5e62535147bca4bf736faaf0f227-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6cce5e62535147bca4bf736faaf0f227 columnFamilyName family1 2023-03-10 18:55:43,049 INFO [StoreOpener-6cce5e62535147bca4bf736faaf0f227-1] regionserver.HStore(310): Store=6cce5e62535147bca4bf736faaf0f227/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:43,049 INFO [StoreOpener-6cce5e62535147bca4bf736faaf0f227-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6cce5e62535147bca4bf736faaf0f227 2023-03-10 18:55:43,050 DEBUG [StoreOpener-6cce5e62535147bca4bf736faaf0f227-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2 2023-03-10 18:55:43,050 DEBUG [StoreOpener-6cce5e62535147bca4bf736faaf0f227-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2 2023-03-10 18:55:43,051 INFO [StoreOpener-6cce5e62535147bca4bf736faaf0f227-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6cce5e62535147bca4bf736faaf0f227 columnFamilyName family2 2023-03-10 18:55:43,052 INFO [StoreOpener-6cce5e62535147bca4bf736faaf0f227-1] regionserver.HStore(310): Store=6cce5e62535147bca4bf736faaf0f227/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:43,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227 2023-03-10 18:55:43,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227 2023-03-10 18:55:43,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:55:43,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6cce5e62535147bca4bf736faaf0f227 2023-03-10 18:55:43,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:55:43,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6cce5e62535147bca4bf736faaf0f227; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72191496, jitterRate=0.0757371187210083}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:55:43,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6cce5e62535147bca4bf736faaf0f227: 2023-03-10 18:55:43,061 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): e46cb331129008feebe80244602e4706#family2#compaction#118 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:43,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227., pid=109, masterSystemTime=1678474543042 2023-03-10 18:55:43,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:43,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:43,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:43,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 86d4ba0f8509f65063fb59feedda6774, NAME => 'ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774.', STARTKEY => '2', ENDKEY => ''} 2023-03-10 18:55:43,064 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=6cce5e62535147bca4bf736faaf0f227, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:43,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 86d4ba0f8509f65063fb59feedda6774 2023-03-10 18:55:43,065 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678474543064"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474543064"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474543064"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474543064"}]},"ts":"1678474543064"} 2023-03-10 18:55:43,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:43,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 86d4ba0f8509f65063fb59feedda6774 2023-03-10 18:55:43,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 86d4ba0f8509f65063fb59feedda6774 2023-03-10 18:55:43,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=106 2023-03-10 18:55:43,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=106, state=SUCCESS; OpenRegionProcedure 6cce5e62535147bca4bf736faaf0f227, server=jenkins-hbase4.apache.org,43181,1678474414959 in 176 msec 2023-03-10 18:55:43,068 INFO [StoreOpener-86d4ba0f8509f65063fb59feedda6774-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 86d4ba0f8509f65063fb59feedda6774 2023-03-10 18:55:43,068 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6cce5e62535147bca4bf736faaf0f227, ASSIGN in 335 msec 2023-03-10 18:55:43,073 DEBUG [StoreOpener-86d4ba0f8509f65063fb59feedda6774-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1 2023-03-10 18:55:43,073 DEBUG [StoreOpener-86d4ba0f8509f65063fb59feedda6774-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1 2023-03-10 18:55:43,073 INFO [StoreOpener-86d4ba0f8509f65063fb59feedda6774-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86d4ba0f8509f65063fb59feedda6774 columnFamilyName family1 2023-03-10 18:55:43,074 INFO [StoreOpener-86d4ba0f8509f65063fb59feedda6774-1] regionserver.HStore(310): Store=86d4ba0f8509f65063fb59feedda6774/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:43,075 INFO [StoreOpener-86d4ba0f8509f65063fb59feedda6774-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 86d4ba0f8509f65063fb59feedda6774 2023-03-10 18:55:43,076 DEBUG [StoreOpener-86d4ba0f8509f65063fb59feedda6774-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2 2023-03-10 18:55:43,076 DEBUG [StoreOpener-86d4ba0f8509f65063fb59feedda6774-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2 2023-03-10 18:55:43,076 INFO [StoreOpener-86d4ba0f8509f65063fb59feedda6774-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86d4ba0f8509f65063fb59feedda6774 columnFamilyName family2 2023-03-10 18:55:43,077 INFO [StoreOpener-86d4ba0f8509f65063fb59feedda6774-1] regionserver.HStore(310): Store=86d4ba0f8509f65063fb59feedda6774/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:43,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774 2023-03-10 18:55:43,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774 2023-03-10 18:55:43,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:55:43,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 86d4ba0f8509f65063fb59feedda6774 2023-03-10 18:55:43,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:55:43,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 86d4ba0f8509f65063fb59feedda6774; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70675578, jitterRate=0.05314818024635315}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:55:43,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 86d4ba0f8509f65063fb59feedda6774: 2023-03-10 18:55:43,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774., pid=111, masterSystemTime=1678474543042 2023-03-10 18:55:43,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:43,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:43,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:43,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3e4f64f1240b6c6abb2116376732f633, NAME => 'ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633.', STARTKEY => '1', ENDKEY => '2'} 2023-03-10 18:55:43,088 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=86d4ba0f8509f65063fb59feedda6774, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:43,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 3e4f64f1240b6c6abb2116376732f633 2023-03-10 18:55:43,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:55:43,088 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678474543087"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474543087"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474543087"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474543087"}]},"ts":"1678474543087"} 2023-03-10 18:55:43,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3e4f64f1240b6c6abb2116376732f633 2023-03-10 18:55:43,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3e4f64f1240b6c6abb2116376732f633 2023-03-10 18:55:43,089 INFO [StoreOpener-3e4f64f1240b6c6abb2116376732f633-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3e4f64f1240b6c6abb2116376732f633 2023-03-10 18:55:43,090 DEBUG [StoreOpener-3e4f64f1240b6c6abb2116376732f633-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1 2023-03-10 18:55:43,090 DEBUG [StoreOpener-3e4f64f1240b6c6abb2116376732f633-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1 2023-03-10 18:55:43,090 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=108 2023-03-10 18:55:43,090 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure 86d4ba0f8509f65063fb59feedda6774, server=jenkins-hbase4.apache.org,43181,1678474414959 in 198 msec 2023-03-10 18:55:43,091 INFO [StoreOpener-3e4f64f1240b6c6abb2116376732f633-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3e4f64f1240b6c6abb2116376732f633 columnFamilyName family1 2023-03-10 18:55:43,091 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=86d4ba0f8509f65063fb59feedda6774, ASSIGN in 358 msec 2023-03-10 18:55:43,091 INFO [StoreOpener-3e4f64f1240b6c6abb2116376732f633-1] regionserver.HStore(310): Store=3e4f64f1240b6c6abb2116376732f633/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:43,092 INFO [StoreOpener-3e4f64f1240b6c6abb2116376732f633-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3e4f64f1240b6c6abb2116376732f633 2023-03-10 18:55:43,093 DEBUG [StoreOpener-3e4f64f1240b6c6abb2116376732f633-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2 2023-03-10 18:55:43,093 DEBUG [StoreOpener-3e4f64f1240b6c6abb2116376732f633-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2 2023-03-10 18:55:43,093 INFO [StoreOpener-3e4f64f1240b6c6abb2116376732f633-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3e4f64f1240b6c6abb2116376732f633 columnFamilyName family2 2023-03-10 18:55:43,094 INFO [StoreOpener-3e4f64f1240b6c6abb2116376732f633-1] regionserver.HStore(310): Store=3e4f64f1240b6c6abb2116376732f633/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:55:43,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633 2023-03-10 18:55:43,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633 2023-03-10 18:55:43,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:55:43,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3e4f64f1240b6c6abb2116376732f633 2023-03-10 18:55:43,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:55:43,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3e4f64f1240b6c6abb2116376732f633; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65913209, jitterRate=-0.01781664788722992}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:55:43,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3e4f64f1240b6c6abb2116376732f633: 2023-03-10 18:55:43,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633., pid=110, masterSystemTime=1678474543042 2023-03-10 18:55:43,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:43,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:43,100 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=3e4f64f1240b6c6abb2116376732f633, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:43,100 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678474543100"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474543100"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474543100"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474543100"}]},"ts":"1678474543100"} 2023-03-10 18:55:43,102 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=107 2023-03-10 18:55:43,102 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=107, state=SUCCESS; OpenRegionProcedure 3e4f64f1240b6c6abb2116376732f633, server=jenkins-hbase4.apache.org,43181,1678474414959 in 210 msec 2023-03-10 18:55:43,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=105 2023-03-10 18:55:43,104 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3e4f64f1240b6c6abb2116376732f633, ASSIGN in 370 msec 2023-03-10 18:55:43,104 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:55:43,104 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474543104"}]},"ts":"1678474543104"} 2023-03-10 18:55:43,105 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-10 18:55:43,107 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:55:43,108 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2150 sec 2023-03-10 18:55:43,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-960884e555924219b3af19c08d09a0fd, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-c264e685deff465a9024c52427e64a02, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-972cf7551eb24a32b6db1f98c0be0339, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-49b0c444247a463e94f4df031173ec97, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/bbd92dea2ef64bdaa5064782ca107d0b_SeqId_34_] to archive 2023-03-10 18:55:43,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:43,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-960884e555924219b3af19c08d09a0fd to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-960884e555924219b3af19c08d09a0fd 2023-03-10 18:55:43,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-c264e685deff465a9024c52427e64a02 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-c264e685deff465a9024c52427e64a02 2023-03-10 18:55:43,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-972cf7551eb24a32b6db1f98c0be0339 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-972cf7551eb24a32b6db1f98c0be0339 2023-03-10 18:55:43,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-49b0c444247a463e94f4df031173ec97 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-49b0c444247a463e94f4df031173ec97 2023-03-10 18:55:43,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/bbd92dea2ef64bdaa5064782ca107d0b_SeqId_34_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family1/bbd92dea2ef64bdaa5064782ca107d0b_SeqId_34_ 2023-03-10 18:55:43,883 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/.tmp/family2/17a3c75041e54b9e99a0b2bac8cff7dd as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/17a3c75041e54b9e99a0b2bac8cff7dd 2023-03-10 18:55:43,887 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in b1d206b11bc34c2bde324dcef1ee6a89/family2 of b1d206b11bc34c2bde324dcef1ee6a89 into 17a3c75041e54b9e99a0b2bac8cff7dd(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:43,887 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:55:43,887 INFO [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89., storeName=b1d206b11bc34c2bde324dcef1ee6a89/family2, priority=12, startTime=1678474540603; duration=1sec 2023-03-10 18:55:43,887 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-10 18:55:43,887 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-10 18:55:43,888 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-10 18:55:43,888 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-10 18:55:43,888 DEBUG [RS:0;jenkins-hbase4:43181-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. because compaction request was cancelled 2023-03-10 18:55:43,889 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/.tmp/family2/f2cbcc8f83724aba9671846e0f4ba6f0 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/f2cbcc8f83724aba9671846e0f4ba6f0 2023-03-10 18:55:43,894 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in e46cb331129008feebe80244602e4706/family2 of e46cb331129008feebe80244602e4706 into f2cbcc8f83724aba9671846e0f4ba6f0(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:55:43,894 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:55:43,894 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706., storeName=e46cb331129008feebe80244602e4706/family2, priority=12, startTime=1678474540656; duration=0sec 2023-03-10 18:55:43,894 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:55:43,910 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-ac6f63f68c4f4410b8660234c52ecb68, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-33c96e7f08cf4a1594f5c4700912b827, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-74182c20bf50441cbffc56e7b1fbc79f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-9654f0864397433d9743a6560de07ee3] to archive 2023-03-10 18:55:43,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:43,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-ac6f63f68c4f4410b8660234c52ecb68 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-ac6f63f68c4f4410b8660234c52ecb68 2023-03-10 18:55:43,914 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-33c96e7f08cf4a1594f5c4700912b827 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-33c96e7f08cf4a1594f5c4700912b827 2023-03-10 18:55:43,915 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-74182c20bf50441cbffc56e7b1fbc79f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-74182c20bf50441cbffc56e7b1fbc79f 2023-03-10 18:55:43,916 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-9654f0864397433d9743a6560de07ee3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/family2/testCompactionWithHFileLink=e46cb331129008feebe80244602e4706-9654f0864397433d9743a6560de07ee3 2023-03-10 18:55:43,918 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-d67d818c37f74d62a3582f788f5c5e93, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0c40ba5afb67448695dd46892c3b5bad, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-f4f5442a622542db857adf9fdfb3f80b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-4ced761ac1c0490980f2d865fa31a9e2] to archive 2023-03-10 18:55:43,918 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:43,920 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-d67d818c37f74d62a3582f788f5c5e93 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-d67d818c37f74d62a3582f788f5c5e93 2023-03-10 18:55:43,921 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0c40ba5afb67448695dd46892c3b5bad to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-0c40ba5afb67448695dd46892c3b5bad 2023-03-10 18:55:43,922 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-f4f5442a622542db857adf9fdfb3f80b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-f4f5442a622542db857adf9fdfb3f80b 2023-03-10 18:55:43,923 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-4ced761ac1c0490980f2d865fa31a9e2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/family2/testCompactionWithHFileLink=b1d206b11bc34c2bde324dcef1ee6a89-4ced761ac1c0490980f2d865fa31a9e2 2023-03-10 18:55:44,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-10 18:55:44,002 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-10 18:55:44,007 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:44,008 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:44,009 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:44,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:44,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-10 18:55:44,012 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-10 18:55:44,012 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:44,012 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-10 18:55:44,012 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-10 18:55:44,013 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:44,013 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:44,014 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:44,014 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:44,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:44,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:44,015 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:44,015 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:44,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:44,015 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:44,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:44,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:44,015 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-10 18:55:44,016 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-10 18:55:44,016 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-10 18:55:44,016 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:44,016 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-10 18:55:44,016 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:44,016 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. started... 2023-03-10 18:55:44,018 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6cce5e62535147bca4bf736faaf0f227 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-10 18:55:44,018 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:44,018 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. started... 2023-03-10 18:55:44,018 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 86d4ba0f8509f65063fb59feedda6774 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-10 18:55:44,018 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:55:44,019 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:44,019 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:44,019 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. started... 2023-03-10 18:55:44,019 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 3e4f64f1240b6c6abb2116376732f633 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-10 18:55:44,433 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303109837794828184f59b764b721b1aaf7f7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303109837794828184f59b764b721b1aaf7f7 2023-03-10 18:55:44,433 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303103158be288d7a481399aa6f4bfb880cfa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303103158be288d7a481399aa6f4bfb880cfa 2023-03-10 18:55:44,433 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310366b079de2e9418aa4c02a3a389c1db9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230310366b079de2e9418aa4c02a3a389c1db9 2023-03-10 18:55:44,838 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family1/0c55400bbbb74079a24662a1802d9ff6 2023-03-10 18:55:44,838 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family1/08e65e3f8bb54dd8a50897c15b2b8afa 2023-03-10 18:55:44,838 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family1/1cb77d2862524774b9a862c42385a26f 2023-03-10 18:55:45,253 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310f72c9c734e7845b6a25b4996d1fb8445 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230310f72c9c734e7845b6a25b4996d1fb8445 2023-03-10 18:55:45,253 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031060891bd9fd7045129baa6a2a3cac297d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023031060891bd9fd7045129baa6a2a3cac297d 2023-03-10 18:55:45,257 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310d1905fae22a64c358f010ad67c8591f6 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230310d1905fae22a64c358f010ad67c8591f6 2023-03-10 18:55:45,260 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family2/ddfb3cf46d0b45e9bbbbdd99b4295639 2023-03-10 18:55:45,260 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family2/642fdd75ecec4371a15395bd1f1d5ab1 2023-03-10 18:55:45,264 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family1/0c55400bbbb74079a24662a1802d9ff6 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/0c55400bbbb74079a24662a1802d9ff6 2023-03-10 18:55:45,264 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family1/08e65e3f8bb54dd8a50897c15b2b8afa as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/08e65e3f8bb54dd8a50897c15b2b8afa 2023-03-10 18:55:45,267 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/0c55400bbbb74079a24662a1802d9ff6, entries=4, sequenceid=5, filesize=5.4 K 2023-03-10 18:55:45,267 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/08e65e3f8bb54dd8a50897c15b2b8afa, entries=4, sequenceid=5, filesize=5.4 K 2023-03-10 18:55:45,268 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family2/642fdd75ecec4371a15395bd1f1d5ab1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/642fdd75ecec4371a15395bd1f1d5ab1 2023-03-10 18:55:45,268 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family2/ddfb3cf46d0b45e9bbbbdd99b4295639 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/ddfb3cf46d0b45e9bbbbdd99b4295639 2023-03-10 18:55:45,269 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family2/1efc4e36b0d34779a1e4990ad42d4f25 2023-03-10 18:55:45,272 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/642fdd75ecec4371a15395bd1f1d5ab1, entries=2, sequenceid=5, filesize=5.1 K 2023-03-10 18:55:45,272 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/ddfb3cf46d0b45e9bbbbdd99b4295639, entries=2, sequenceid=5, filesize=5.1 K 2023-03-10 18:55:45,273 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family1/1cb77d2862524774b9a862c42385a26f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/1cb77d2862524774b9a862c42385a26f 2023-03-10 18:55:45,273 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6cce5e62535147bca4bf736faaf0f227 in 1255ms, sequenceid=5, compaction requested=false 2023-03-10 18:55:45,273 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 86d4ba0f8509f65063fb59feedda6774 in 1255ms, sequenceid=5, compaction requested=false 2023-03-10 18:55:45,273 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-10 18:55:45,273 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-10 18:55:45,274 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6cce5e62535147bca4bf736faaf0f227: 2023-03-10 18:55:45,274 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 86d4ba0f8509f65063fb59feedda6774: 2023-03-10 18:55:45,275 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:45,275 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:55:45,275 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:45,275 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:55:45,277 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/1cb77d2862524774b9a862c42385a26f, entries=4, sequenceid=5, filesize=5.4 K 2023-03-10 18:55:45,278 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family2/1efc4e36b0d34779a1e4990ad42d4f25 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/1efc4e36b0d34779a1e4990ad42d4f25 2023-03-10 18:55:45,281 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/1efc4e36b0d34779a1e4990ad42d4f25, entries=2, sequenceid=5, filesize=5.1 K 2023-03-10 18:55:45,282 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 3e4f64f1240b6c6abb2116376732f633 in 1263ms, sequenceid=5, compaction requested=false 2023-03-10 18:55:45,282 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 3e4f64f1240b6c6abb2116376732f633: 2023-03-10 18:55:45,282 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:55:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:55:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-10 18:55:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-10 18:55:45,284 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,284 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:45,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:45,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:45,284 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:45,284 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:45,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:45,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:45,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-10 18:55:45,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@748ac672[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:45,286 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-10 18:55:45,286 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:45,288 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:45,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:45,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:45,288 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-10 18:55:45,288 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-10 18:55:45,288 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:45,288 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,289 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:45,290 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-10 18:55:45,290 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:45,290 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,290 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-10 18:55:45,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:45,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:45,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:45,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:45,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:45,293 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-10 18:55:45,293 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-10 18:55:45,293 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:45,293 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:45,293 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-10 18:55:45,293 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:45,294 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:45,294 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:45,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:45,294 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:45,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:45,295 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:45,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:45,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,298 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:45,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:45,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,301 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:45,301 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:45,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:45,301 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:45,301 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:45,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-10 18:55:45,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:45,301 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:45,301 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:45,301 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:45,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:45,301 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:45,301 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:45,302 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:45,301 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:45,302 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:45,402 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:45,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:45,404 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:45,404 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:45,404 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:45,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:45,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-10 18:55:45,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-10 18:55:45,408 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-10 18:55:45,408 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:45,408 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-10 18:55:45,408 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-10 18:55:45,409 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:45,409 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:45,411 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,411 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:45,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:45,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:45,411 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:45,411 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:45,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:45,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:45,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:45,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:45,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-10 18:55:45,412 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-10 18:55:45,412 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-10 18:55:45,412 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-10 18:55:45,412 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:45,412 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-10 18:55:45,412 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:45,412 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. started... 2023-03-10 18:55:45,412 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:55:45,413 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6cce5e62535147bca4bf736faaf0f227 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-10 18:55:45,413 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:45,412 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:45,413 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:45,413 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. started... 2023-03-10 18:55:45,413 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. started... 2023-03-10 18:55:45,413 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 86d4ba0f8509f65063fb59feedda6774 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-10 18:55:45,413 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 3e4f64f1240b6c6abb2116376732f633 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-10 18:55:45,428 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310fb4ea8c1106c4335b4f57148098a6ed9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230310fb4ea8c1106c4335b4f57148098a6ed9 2023-03-10 18:55:45,433 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family1/6dc195ec4bbc478a8e9db69620eddbbb 2023-03-10 18:55:45,829 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310ea423804b8e0433cb10be04b009240d7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230310ea423804b8e0433cb10be04b009240d7 2023-03-10 18:55:45,829 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303104bf958f8e6a247799756c2272254ec16 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303104bf958f8e6a247799756c2272254ec16 2023-03-10 18:55:45,845 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023031053882b79cad945d680ca06c0a69e8c34 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023031053882b79cad945d680ca06c0a69e8c34 2023-03-10 18:55:45,849 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family2/68e7a146039c46139d8ceff3da67951f 2023-03-10 18:55:45,852 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family1/6dc195ec4bbc478a8e9db69620eddbbb as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/6dc195ec4bbc478a8e9db69620eddbbb 2023-03-10 18:55:45,855 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/6dc195ec4bbc478a8e9db69620eddbbb, entries=4, sequenceid=9, filesize=5.4 K 2023-03-10 18:55:45,855 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family2/68e7a146039c46139d8ceff3da67951f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/68e7a146039c46139d8ceff3da67951f 2023-03-10 18:55:45,858 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/68e7a146039c46139d8ceff3da67951f, entries=2, sequenceid=9, filesize=5.1 K 2023-03-10 18:55:45,859 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 86d4ba0f8509f65063fb59feedda6774 in 446ms, sequenceid=9, compaction requested=false 2023-03-10 18:55:45,859 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 86d4ba0f8509f65063fb59feedda6774: 2023-03-10 18:55:45,859 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:45,859 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:55:46,234 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family1/cd590da4656f4d5db26c8716612ff7ac 2023-03-10 18:55:46,234 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family1/746e714f63f64242b5219e6d888f5596 2023-03-10 18:55:46,252 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031072e2d9fc481b4581b53c2200829b4568 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023031072e2d9fc481b4581b53c2200829b4568 2023-03-10 18:55:46,653 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310d36ddf351da1496998ef73b7d018a981 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230310d36ddf351da1496998ef73b7d018a981 2023-03-10 18:55:46,661 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family2/6760ed972deb4788933982fcc658e425 2023-03-10 18:55:46,665 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family1/746e714f63f64242b5219e6d888f5596 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/746e714f63f64242b5219e6d888f5596 2023-03-10 18:55:46,669 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/746e714f63f64242b5219e6d888f5596, entries=4, sequenceid=9, filesize=5.4 K 2023-03-10 18:55:46,670 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family2/6760ed972deb4788933982fcc658e425 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/6760ed972deb4788933982fcc658e425 2023-03-10 18:55:46,673 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/6760ed972deb4788933982fcc658e425, entries=2, sequenceid=9, filesize=5.1 K 2023-03-10 18:55:46,674 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6cce5e62535147bca4bf736faaf0f227 in 1262ms, sequenceid=9, compaction requested=false 2023-03-10 18:55:46,674 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6cce5e62535147bca4bf736faaf0f227: 2023-03-10 18:55:46,674 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:46,674 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:55:47,058 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family2/553bb13ab0754db49e803799daca7c4d 2023-03-10 18:55:47,062 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family1/cd590da4656f4d5db26c8716612ff7ac as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/cd590da4656f4d5db26c8716612ff7ac 2023-03-10 18:55:47,065 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/cd590da4656f4d5db26c8716612ff7ac, entries=4, sequenceid=9, filesize=5.4 K 2023-03-10 18:55:47,066 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family2/553bb13ab0754db49e803799daca7c4d as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/553bb13ab0754db49e803799daca7c4d 2023-03-10 18:55:47,070 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/553bb13ab0754db49e803799daca7c4d, entries=2, sequenceid=9, filesize=5.1 K 2023-03-10 18:55:47,070 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 3e4f64f1240b6c6abb2116376732f633 in 1657ms, sequenceid=9, compaction requested=false 2023-03-10 18:55:47,071 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 3e4f64f1240b6c6abb2116376732f633: 2023-03-10 18:55:47,071 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:47,071 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:55:47,071 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:55:47,071 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,071 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-10 18:55:47,071 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-10 18:55:47,072 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,072 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:47,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:47,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:47,073 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:47,073 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:47,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:47,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:47,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:47,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:47,074 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-10 18:55:47,074 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30f02e80[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:47,074 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-10 18:55:47,074 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:47,075 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:47,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:47,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:47,075 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-10 18:55:47,075 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-10 18:55:47,075 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:47,075 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,075 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:47,080 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,080 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-10 18:55:47,080 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:47,080 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-10 18:55:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:47,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:47,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:47,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:47,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:47,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:47,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-10 18:55:47,082 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-10 18:55:47,082 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:47,082 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:47,082 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-10 18:55:47,082 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:47,084 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:47,084 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:47,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:47,084 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:47,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:47,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:47,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:47,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:47,084 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:47,085 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:47,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:47,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:47,087 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:47,087 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,087 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:47,087 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:47,087 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:47,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:47,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-10 18:55:47,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:47,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:47,087 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,088 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:47,088 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:47,088 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:47,087 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:47,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-10 18:55:47,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:47,088 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:47,088 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:47,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:47,188 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:47,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:47,190 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:47,190 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:47,190 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:47,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:47,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-10 18:55:47,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-10 18:55:47,194 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-10 18:55:47,194 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:47,194 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-10 18:55:47,194 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-10 18:55:47,195 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:47,195 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:47,196 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:47,196 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:47,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:47,197 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:47,197 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:47,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:47,197 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:47,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:47,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:47,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-10 18:55:47,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-10 18:55:47,198 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-10 18:55:47,198 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-10 18:55:47,198 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:47,198 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-10 18:55:47,198 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:47,198 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. started... 2023-03-10 18:55:47,198 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:47,199 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. started... 2023-03-10 18:55:47,199 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6cce5e62535147bca4bf736faaf0f227 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-10 18:55:47,199 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 86d4ba0f8509f65063fb59feedda6774 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-10 18:55:47,199 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:55:47,199 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:47,199 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:47,199 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. started... 2023-03-10 18:55:47,199 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 3e4f64f1240b6c6abb2116376732f633 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-10 18:55:47,618 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303104f379cf4f8e94eb1bf0b443819638152 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303104f379cf4f8e94eb1bf0b443819638152 2023-03-10 18:55:47,624 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310eca469a8fc284f19905592f1cf88b00e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230310eca469a8fc284f19905592f1cf88b00e 2023-03-10 18:55:47,626 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310d81ff1a1411e4249bb3dc25042c37b47 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230310d81ff1a1411e4249bb3dc25042c37b47 2023-03-10 18:55:47,632 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family1/d4aa15e6b24c4a338c9d3efca95d4e9a 2023-03-10 18:55:48,024 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family1/326e58cf1e7e4ff6a1bdc40ce22034b7 2023-03-10 18:55:48,032 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family1/11652f75e920468d9a432793d206de24 2023-03-10 18:55:48,049 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310290dcfa34a6c472096a630d446c54322 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230310290dcfa34a6c472096a630d446c54322 2023-03-10 18:55:48,437 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310a62fe32d2f544d75987bd8d00df6db28 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230310a62fe32d2f544d75987bd8d00df6db28 2023-03-10 18:55:48,444 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310253093938c484711a5c9e6282bc86f18 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230310253093938c484711a5c9e6282bc86f18 2023-03-10 18:55:48,453 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family2/f148abd99ca0429c826c6a37b9b5822c 2023-03-10 18:55:48,456 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family1/d4aa15e6b24c4a338c9d3efca95d4e9a as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/d4aa15e6b24c4a338c9d3efca95d4e9a 2023-03-10 18:55:48,459 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/d4aa15e6b24c4a338c9d3efca95d4e9a, entries=4, sequenceid=13, filesize=5.4 K 2023-03-10 18:55:48,460 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family2/f148abd99ca0429c826c6a37b9b5822c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/f148abd99ca0429c826c6a37b9b5822c 2023-03-10 18:55:48,463 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/f148abd99ca0429c826c6a37b9b5822c, entries=2, sequenceid=13, filesize=5.1 K 2023-03-10 18:55:48,464 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 86d4ba0f8509f65063fb59feedda6774 in 1265ms, sequenceid=13, compaction requested=true 2023-03-10 18:55:48,464 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 86d4ba0f8509f65063fb59feedda6774: 2023-03-10 18:55:48,464 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:55:48,546 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:55:48,842 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family2/34ee820edbbf487c8286ee1b2c11d314 2023-03-10 18:55:48,847 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family1/326e58cf1e7e4ff6a1bdc40ce22034b7 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/326e58cf1e7e4ff6a1bdc40ce22034b7 2023-03-10 18:55:48,849 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family2/9f75b16d47b043fba506d9eb228f6ad4 2023-03-10 18:55:48,850 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/326e58cf1e7e4ff6a1bdc40ce22034b7, entries=4, sequenceid=13, filesize=5.4 K 2023-03-10 18:55:48,851 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family2/34ee820edbbf487c8286ee1b2c11d314 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/34ee820edbbf487c8286ee1b2c11d314 2023-03-10 18:55:48,852 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family1/11652f75e920468d9a432793d206de24 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/11652f75e920468d9a432793d206de24 2023-03-10 18:55:48,854 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/34ee820edbbf487c8286ee1b2c11d314, entries=2, sequenceid=13, filesize=5.1 K 2023-03-10 18:55:48,855 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 3e4f64f1240b6c6abb2116376732f633 in 1656ms, sequenceid=13, compaction requested=true 2023-03-10 18:55:48,855 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 3e4f64f1240b6c6abb2116376732f633: 2023-03-10 18:55:48,855 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:48,855 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:55:48,856 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/11652f75e920468d9a432793d206de24, entries=4, sequenceid=13, filesize=5.4 K 2023-03-10 18:55:48,856 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family2/9f75b16d47b043fba506d9eb228f6ad4 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f75b16d47b043fba506d9eb228f6ad4 2023-03-10 18:55:48,859 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f75b16d47b043fba506d9eb228f6ad4, entries=2, sequenceid=13, filesize=5.1 K 2023-03-10 18:55:48,860 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6cce5e62535147bca4bf736faaf0f227 in 1661ms, sequenceid=13, compaction requested=true 2023-03-10 18:55:48,860 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6cce5e62535147bca4bf736faaf0f227: 2023-03-10 18:55:48,860 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:48,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:55:48,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:55:48,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-10 18:55:48,860 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-10 18:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:48,863 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:48,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:48,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:48,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-10 18:55:48,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7555eab[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:48,864 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-10 18:55:48,864 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:48,866 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,866 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:48,866 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:48,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:48,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:48,866 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-10 18:55:48,866 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-10 18:55:48,866 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:48,868 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-10 18:55:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-10 18:55:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:48,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-10 18:55:48,870 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-10 18:55:48,870 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:48,870 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:48,870 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-10 18:55:48,870 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:48,871 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:48,871 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:48,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:48,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:48,872 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:48,872 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:48,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:48,873 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:48,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:48,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-10 18:55:48,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-10 18:55:48,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:48,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:48,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:48,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:48,876 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:48,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:48,876 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:48,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:48,876 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:48,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:48,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:48,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:48,876 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:48,976 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:48,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:48,978 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:48,978 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:48,978 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:55:48,980 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:48,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-10 18:55:48,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-10 18:55:48,982 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-10 18:55:48,982 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:48,982 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-10 18:55:48,982 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-10 18:55:48,983 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:48,983 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:48,984 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,984 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:48,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:48,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:48,984 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:48,984 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:48,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:48,985 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:48,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:48,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:48,985 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-10 18:55:48,985 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-10 18:55:48,985 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-10 18:55:48,985 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-10 18:55:48,985 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:48,985 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-10 18:55:48,986 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:48,986 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. started... 2023-03-10 18:55:48,986 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:55:48,986 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:48,986 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6cce5e62535147bca4bf736faaf0f227 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-10 18:55:48,986 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:48,986 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. started... 2023-03-10 18:55:48,986 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. started... 2023-03-10 18:55:48,986 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:48,986 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 86d4ba0f8509f65063fb59feedda6774 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-10 18:55:48,986 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 3e4f64f1240b6c6abb2116376732f633 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-10 18:55:49,402 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303109abed8a121bd403cbf660733944433d7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303109abed8a121bd403cbf660733944433d7 2023-03-10 18:55:49,402 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023031036e496d9bd374ccb8231b217d03b5a70 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023031036e496d9bd374ccb8231b217d03b5a70 2023-03-10 18:55:49,402 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310e1109df7b5a94285b19f157d93cdbc9c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230310e1109df7b5a94285b19f157d93cdbc9c 2023-03-10 18:55:49,807 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family1/796ab348fa294d2ba0a9fc7cc9d3d85f 2023-03-10 18:55:49,807 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family1/5c6b23f4a6a54e8dadcd9d4f598fa693 2023-03-10 18:55:49,807 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family1/325a357e20dd45ec92c2702bc810f0db 2023-03-10 18:55:49,822 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310ef284717e797444f8c66280ce75d0109 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230310ef284717e797444f8c66280ce75d0109 2023-03-10 18:55:50,223 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310a6bb1f96a3bd49c4b61a7691fa112f66 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230310a6bb1f96a3bd49c4b61a7691fa112f66 2023-03-10 18:55:50,223 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023031059176be6ad2a40eeab3cd20996d79711 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023031059176be6ad2a40eeab3cd20996d79711 2023-03-10 18:55:50,226 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family2/9f4a7cffb2f94e18bb43e9f02004cb1b 2023-03-10 18:55:50,228 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family2/f20ce428f7d647f2afa08e4a621c3e22 2023-03-10 18:55:50,230 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family1/325a357e20dd45ec92c2702bc810f0db as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/325a357e20dd45ec92c2702bc810f0db 2023-03-10 18:55:50,231 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family1/5c6b23f4a6a54e8dadcd9d4f598fa693 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/5c6b23f4a6a54e8dadcd9d4f598fa693 2023-03-10 18:55:50,233 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/325a357e20dd45ec92c2702bc810f0db, entries=4, sequenceid=17, filesize=5.4 K 2023-03-10 18:55:50,234 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family2/9f4a7cffb2f94e18bb43e9f02004cb1b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f4a7cffb2f94e18bb43e9f02004cb1b 2023-03-10 18:55:50,234 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/5c6b23f4a6a54e8dadcd9d4f598fa693, entries=4, sequenceid=17, filesize=5.4 K 2023-03-10 18:55:50,235 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family2/f20ce428f7d647f2afa08e4a621c3e22 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/f20ce428f7d647f2afa08e4a621c3e22 2023-03-10 18:55:50,241 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f4a7cffb2f94e18bb43e9f02004cb1b, entries=2, sequenceid=17, filesize=5.1 K 2023-03-10 18:55:50,241 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6cce5e62535147bca4bf736faaf0f227 in 1255ms, sequenceid=17, compaction requested=true 2023-03-10 18:55:50,241 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6cce5e62535147bca4bf736faaf0f227: 2023-03-10 18:55:50,241 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:50,241 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:55:50,242 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/f20ce428f7d647f2afa08e4a621c3e22, entries=2, sequenceid=17, filesize=5.1 K 2023-03-10 18:55:50,242 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 3e4f64f1240b6c6abb2116376732f633 in 1256ms, sequenceid=17, compaction requested=true 2023-03-10 18:55:50,242 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 3e4f64f1240b6c6abb2116376732f633: 2023-03-10 18:55:50,242 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:50,243 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:55:50,628 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family2/9a8b6b79f6444ef18ef8261ea25c9967 2023-03-10 18:55:50,633 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family1/796ab348fa294d2ba0a9fc7cc9d3d85f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/796ab348fa294d2ba0a9fc7cc9d3d85f 2023-03-10 18:55:50,636 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/796ab348fa294d2ba0a9fc7cc9d3d85f, entries=4, sequenceid=17, filesize=5.4 K 2023-03-10 18:55:50,636 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family2/9a8b6b79f6444ef18ef8261ea25c9967 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/9a8b6b79f6444ef18ef8261ea25c9967 2023-03-10 18:55:50,640 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/9a8b6b79f6444ef18ef8261ea25c9967, entries=2, sequenceid=17, filesize=5.1 K 2023-03-10 18:55:50,641 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 86d4ba0f8509f65063fb59feedda6774 in 1654ms, sequenceid=17, compaction requested=true 2023-03-10 18:55:50,641 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 86d4ba0f8509f65063fb59feedda6774: 2023-03-10 18:55:50,641 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:50,641 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:55:50,641 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:55:50,641 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,641 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-10 18:55:50,641 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-10 18:55:50,642 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:50,642 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:50,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:50,643 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:50,643 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:50,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:50,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:50,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:50,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:50,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-10 18:55:50,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25a4f77f[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:50,644 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-10 18:55:50,644 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:50,645 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:50,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:50,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:50,645 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-10 18:55:50,645 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-10 18:55:50,645 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:50,645 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,646 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:50,648 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,648 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-10 18:55:50,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,648 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:50,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:50,648 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-10 18:55:50,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:50,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:50,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:50,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:50,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:50,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:50,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:50,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-10 18:55:50,650 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-10 18:55:50,650 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:50,650 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:50,650 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-10 18:55:50,650 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:50,652 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:50,652 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:50,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:50,652 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:50,652 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:50,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:50,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:50,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:50,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:50,653 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:50,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:50,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,655 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,655 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:50,655 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:50,655 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,655 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:50,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-10 18:55:50,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:50,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:50,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:50,655 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:50,655 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:50,655 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:50,655 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:50,655 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:50,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:50,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:50,656 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:50,656 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:50,756 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:50,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:50,767 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:50,776 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:50,785 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:50,786 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:50,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:55:50,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-10 18:55:50,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-10 18:55:50,817 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-10 18:55:50,817 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:50,817 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-10 18:55:50,817 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-10 18:55:50,818 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:50,818 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:50,819 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:50,819 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:50,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:50,819 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:50,819 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:55:50,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:50,820 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:50,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:55:50,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:50,820 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-10 18:55:50,820 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-10 18:55:50,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-10 18:55:50,820 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-10 18:55:50,820 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:55:50,820 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-10 18:55:50,821 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:50,821 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. started... 2023-03-10 18:55:50,821 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:50,821 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6cce5e62535147bca4bf736faaf0f227 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-10 18:55:50,821 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:55:50,821 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:55:50,821 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:50,821 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. started... 2023-03-10 18:55:50,821 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. started... 2023-03-10 18:55:50,821 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 3e4f64f1240b6c6abb2116376732f633 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-10 18:55:50,821 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 86d4ba0f8509f65063fb59feedda6774 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-10 18:55:51,250 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family1/b110ef4291d74bc1b91142182f664168 2023-03-10 18:55:51,250 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family1/a7b745089aee46a6a33b6dc84585f4af 2023-03-10 18:55:51,254 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family1/18c1e0d78c4943219d42a9462ffa31fc 2023-03-10 18:55:51,254 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b110ef4291d74bc1b91142182f664168 2023-03-10 18:55:51,254 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7b745089aee46a6a33b6dc84585f4af 2023-03-10 18:55:51,257 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 18c1e0d78c4943219d42a9462ffa31fc 2023-03-10 18:55:51,271 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family2/b17c4e437a0a4a80a80e2413ce18222b 2023-03-10 18:55:51,273 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b17c4e437a0a4a80a80e2413ce18222b 2023-03-10 18:55:51,274 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family1/18c1e0d78c4943219d42a9462ffa31fc as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/18c1e0d78c4943219d42a9462ffa31fc 2023-03-10 18:55:51,277 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 18c1e0d78c4943219d42a9462ffa31fc 2023-03-10 18:55:51,277 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/18c1e0d78c4943219d42a9462ffa31fc, entries=3, sequenceid=23, filesize=5.0 K 2023-03-10 18:55:51,278 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family2/b17c4e437a0a4a80a80e2413ce18222b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/b17c4e437a0a4a80a80e2413ce18222b 2023-03-10 18:55:51,281 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b17c4e437a0a4a80a80e2413ce18222b 2023-03-10 18:55:51,281 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/b17c4e437a0a4a80a80e2413ce18222b, entries=1, sequenceid=23, filesize=5.0 K 2023-03-10 18:55:51,282 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 86d4ba0f8509f65063fb59feedda6774 in 461ms, sequenceid=23, compaction requested=true 2023-03-10 18:55:51,282 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 86d4ba0f8509f65063fb59feedda6774: 2023-03-10 18:55:51,282 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:51,282 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:55:51,669 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family2/8ab5d41a27de44c88a7b215d6fb29dc2 2023-03-10 18:55:51,673 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8ab5d41a27de44c88a7b215d6fb29dc2 2023-03-10 18:55:51,673 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family1/a7b745089aee46a6a33b6dc84585f4af as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/a7b745089aee46a6a33b6dc84585f4af 2023-03-10 18:55:51,676 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7b745089aee46a6a33b6dc84585f4af 2023-03-10 18:55:51,676 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/a7b745089aee46a6a33b6dc84585f4af, entries=3, sequenceid=23, filesize=5.0 K 2023-03-10 18:55:51,677 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family2/8ab5d41a27de44c88a7b215d6fb29dc2 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/8ab5d41a27de44c88a7b215d6fb29dc2 2023-03-10 18:55:51,680 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8ab5d41a27de44c88a7b215d6fb29dc2 2023-03-10 18:55:51,681 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/8ab5d41a27de44c88a7b215d6fb29dc2, entries=1, sequenceid=23, filesize=5.0 K 2023-03-10 18:55:51,681 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6cce5e62535147bca4bf736faaf0f227 in 860ms, sequenceid=23, compaction requested=true 2023-03-10 18:55:51,681 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6cce5e62535147bca4bf736faaf0f227: 2023-03-10 18:55:51,681 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:51,681 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:55:52,071 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family2/923d54907f8d4b1e8b1b6c87103c942c 2023-03-10 18:55:52,074 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 923d54907f8d4b1e8b1b6c87103c942c 2023-03-10 18:55:52,075 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family1/b110ef4291d74bc1b91142182f664168 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/b110ef4291d74bc1b91142182f664168 2023-03-10 18:55:52,078 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b110ef4291d74bc1b91142182f664168 2023-03-10 18:55:52,078 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/b110ef4291d74bc1b91142182f664168, entries=3, sequenceid=23, filesize=5.0 K 2023-03-10 18:55:52,078 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family2/923d54907f8d4b1e8b1b6c87103c942c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/923d54907f8d4b1e8b1b6c87103c942c 2023-03-10 18:55:52,081 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 923d54907f8d4b1e8b1b6c87103c942c 2023-03-10 18:55:52,082 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/923d54907f8d4b1e8b1b6c87103c942c, entries=1, sequenceid=23, filesize=5.0 K 2023-03-10 18:55:52,082 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 3e4f64f1240b6c6abb2116376732f633 in 1261ms, sequenceid=23, compaction requested=true 2023-03-10 18:55:52,082 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 3e4f64f1240b6c6abb2116376732f633: 2023-03-10 18:55:52,082 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:52,082 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:55:52,082 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:55:52,082 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,082 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-10 18:55:52,083 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-10 18:55:52,088 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,088 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:52,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:52,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:52,088 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:52,088 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:55:52,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:52,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:52,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:52,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:52,089 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-10 18:55:52,089 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7bf1cef[Count = 0] remaining members to acquire global barrier 2023-03-10 18:55:52,089 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-10 18:55:52,090 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:52,091 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:52,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:52,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:52,091 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-10 18:55:52,091 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-10 18:55:52,091 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:55:52,091 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,091 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:55:52,092 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,092 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-10 18:55:52,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:52,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:52,092 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:52,092 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-10 18:55:52,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:52,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:52,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:52,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:52,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:52,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:55:52,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-10 18:55:52,094 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-10 18:55:52,094 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:55:52,094 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:55:52,094 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-10 18:55:52,094 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:55:52,102 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:52,102 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:55:52,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:52,102 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:55:52,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:52,102 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:52,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:55:52,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:52,103 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:55:52,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-10 18:55:52,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,110 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,110 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:52,110 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-10 18:55:52,110 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:55:52,110 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:52,110 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:55:52,110 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-10 18:55:52,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:55:52,110 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:55:52,110 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-10 18:55:52,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-10 18:55:52,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:55:52,110 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:55:52,110 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-10 18:55:52,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:55:52,110 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-10 18:55:52,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:55:52,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:55:52,110 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:55:52,211 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:55:52,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:55:52,212 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:52,212 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 6cce5e62535147bca4bf736faaf0f227/family1 is initiating major compaction (all files) 2023-03-10 18:55:52,212 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:52,212 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:52,213 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 6cce5e62535147bca4bf736faaf0f227/family1 in ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:52,213 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/0c55400bbbb74079a24662a1802d9ff6, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/746e714f63f64242b5219e6d888f5596, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/11652f75e920468d9a432793d206de24, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/325a357e20dd45ec92c2702bc810f0db, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/a7b745089aee46a6a33b6dc84585f4af] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp, totalSize=26.6 K 2023-03-10 18:55:52,213 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 0c55400bbbb74079a24662a1802d9ff6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474544007 2023-03-10 18:55:52,214 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 746e714f63f64242b5219e6d888f5596, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474545404 2023-03-10 18:55:52,214 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 11652f75e920468d9a432793d206de24, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474547190 2023-03-10 18:55:52,214 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 325a357e20dd45ec92c2702bc810f0db, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474548978 2023-03-10 18:55:52,215 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting a7b745089aee46a6a33b6dc84585f4af, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:55:52,225 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 6cce5e62535147bca4bf736faaf0f227#family1#compaction#149 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:52,633 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303107a154ca1b8f44c1bb63c2b3d32ee8c87_del 2023-03-10 18:55:52,634 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303107a154ca1b8f44c1bb63c2b3d32ee8c87_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303107a154ca1b8f44c1bb63c2b3d32ee8c87_del 2023-03-10 18:55:53,447 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1ec3c9e4846849e799e91f3afccdb730 2023-03-10 18:55:53,448 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family1/1ec3c9e4846849e799e91f3afccdb730 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/1ec3c9e4846849e799e91f3afccdb730 2023-03-10 18:55:53,451 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1ec3c9e4846849e799e91f3afccdb730 2023-03-10 18:55:53,452 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6cce5e62535147bca4bf736faaf0f227/family1 of 6cce5e62535147bca4bf736faaf0f227 into 1ec3c9e4846849e799e91f3afccdb730(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:53,452 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 6cce5e62535147bca4bf736faaf0f227: 2023-03-10 18:55:53,452 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:53,452 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 6cce5e62535147bca4bf736faaf0f227/family2 is initiating major compaction (all files) 2023-03-10 18:55:53,452 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:53,452 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:53,453 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 6cce5e62535147bca4bf736faaf0f227/family2 in ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:55:53,453 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/ddfb3cf46d0b45e9bbbbdd99b4295639, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/6760ed972deb4788933982fcc658e425, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f75b16d47b043fba506d9eb228f6ad4, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f4a7cffb2f94e18bb43e9f02004cb1b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/8ab5d41a27de44c88a7b215d6fb29dc2] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp, totalSize=25.3 K 2023-03-10 18:55:53,453 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting ddfb3cf46d0b45e9bbbbdd99b4295639, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474544007 2023-03-10 18:55:53,454 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 6760ed972deb4788933982fcc658e425, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474545404 2023-03-10 18:55:53,454 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 9f75b16d47b043fba506d9eb228f6ad4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474547190 2023-03-10 18:55:53,454 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 9f4a7cffb2f94e18bb43e9f02004cb1b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474548978 2023-03-10 18:55:53,454 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 8ab5d41a27de44c88a7b215d6fb29dc2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:55:53,462 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 6cce5e62535147bca4bf736faaf0f227#family2#compaction#150 average throughput is 1.02 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:53,511 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/0c55400bbbb74079a24662a1802d9ff6, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/746e714f63f64242b5219e6d888f5596, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/11652f75e920468d9a432793d206de24, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/325a357e20dd45ec92c2702bc810f0db, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/a7b745089aee46a6a33b6dc84585f4af] to archive 2023-03-10 18:55:53,512 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:53,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/0c55400bbbb74079a24662a1802d9ff6 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/0c55400bbbb74079a24662a1802d9ff6 2023-03-10 18:55:53,515 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/746e714f63f64242b5219e6d888f5596 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/746e714f63f64242b5219e6d888f5596 2023-03-10 18:55:53,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/11652f75e920468d9a432793d206de24 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/11652f75e920468d9a432793d206de24 2023-03-10 18:55:53,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/325a357e20dd45ec92c2702bc810f0db to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/325a357e20dd45ec92c2702bc810f0db 2023-03-10 18:55:53,518 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/a7b745089aee46a6a33b6dc84585f4af to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/a7b745089aee46a6a33b6dc84585f4af 2023-03-10 18:55:53,871 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230310f0e0c5ba51d34426a51a44fdd51d9ed5_del 2023-03-10 18:55:53,871 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310f0e0c5ba51d34426a51a44fdd51d9ed5_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230310f0e0c5ba51d34426a51a44fdd51d9ed5_del 2023-03-10 18:55:54,685 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for abb5515d50b64303b30ff340771a35e2 2023-03-10 18:55:54,685 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/.tmp/family2/abb5515d50b64303b30ff340771a35e2 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/abb5515d50b64303b30ff340771a35e2 2023-03-10 18:55:54,688 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for abb5515d50b64303b30ff340771a35e2 2023-03-10 18:55:54,689 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6cce5e62535147bca4bf736faaf0f227/family2 of 6cce5e62535147bca4bf736faaf0f227 into abb5515d50b64303b30ff340771a35e2(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:54,689 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 6cce5e62535147bca4bf736faaf0f227: 2023-03-10 18:55:54,689 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:54,689 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 86d4ba0f8509f65063fb59feedda6774/family1 is initiating major compaction (all files) 2023-03-10 18:55:54,690 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:54,690 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:54,690 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 86d4ba0f8509f65063fb59feedda6774/family1 in ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:54,690 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/08e65e3f8bb54dd8a50897c15b2b8afa, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/6dc195ec4bbc478a8e9db69620eddbbb, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/d4aa15e6b24c4a338c9d3efca95d4e9a, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/796ab348fa294d2ba0a9fc7cc9d3d85f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/18c1e0d78c4943219d42a9462ffa31fc] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp, totalSize=26.6 K 2023-03-10 18:55:54,690 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 08e65e3f8bb54dd8a50897c15b2b8afa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474544009 2023-03-10 18:55:54,691 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 6dc195ec4bbc478a8e9db69620eddbbb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474545404 2023-03-10 18:55:54,691 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting d4aa15e6b24c4a338c9d3efca95d4e9a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474547190 2023-03-10 18:55:54,691 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 796ab348fa294d2ba0a9fc7cc9d3d85f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474548978 2023-03-10 18:55:54,691 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 18c1e0d78c4943219d42a9462ffa31fc, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:55:54,699 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 86d4ba0f8509f65063fb59feedda6774#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:54,710 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/ddfb3cf46d0b45e9bbbbdd99b4295639, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/6760ed972deb4788933982fcc658e425, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f75b16d47b043fba506d9eb228f6ad4, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f4a7cffb2f94e18bb43e9f02004cb1b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/8ab5d41a27de44c88a7b215d6fb29dc2] to archive 2023-03-10 18:55:54,711 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:54,712 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/ddfb3cf46d0b45e9bbbbdd99b4295639 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/ddfb3cf46d0b45e9bbbbdd99b4295639 2023-03-10 18:55:54,713 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/6760ed972deb4788933982fcc658e425 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/6760ed972deb4788933982fcc658e425 2023-03-10 18:55:54,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f75b16d47b043fba506d9eb228f6ad4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f75b16d47b043fba506d9eb228f6ad4 2023-03-10 18:55:54,715 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f4a7cffb2f94e18bb43e9f02004cb1b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/9f4a7cffb2f94e18bb43e9f02004cb1b 2023-03-10 18:55:54,716 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/8ab5d41a27de44c88a7b215d6fb29dc2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family2/8ab5d41a27de44c88a7b215d6fb29dc2 2023-03-10 18:55:55,108 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230310b1a3efa9898f4fd2b769cbd003846bca_del 2023-03-10 18:55:55,108 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310b1a3efa9898f4fd2b769cbd003846bca_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230310b1a3efa9898f4fd2b769cbd003846bca_del 2023-03-10 18:55:55,921 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c5d7890cef5248b49d8fb619cf5cfa74 2023-03-10 18:55:55,922 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family1/c5d7890cef5248b49d8fb619cf5cfa74 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/c5d7890cef5248b49d8fb619cf5cfa74 2023-03-10 18:55:55,926 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c5d7890cef5248b49d8fb619cf5cfa74 2023-03-10 18:55:55,927 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 86d4ba0f8509f65063fb59feedda6774/family1 of 86d4ba0f8509f65063fb59feedda6774 into c5d7890cef5248b49d8fb619cf5cfa74(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:55,927 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 86d4ba0f8509f65063fb59feedda6774: 2023-03-10 18:55:55,927 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:55,927 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 86d4ba0f8509f65063fb59feedda6774/family2 is initiating major compaction (all files) 2023-03-10 18:55:55,927 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:55,927 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:55,927 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 86d4ba0f8509f65063fb59feedda6774/family2 in ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:55:55,927 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/642fdd75ecec4371a15395bd1f1d5ab1, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/68e7a146039c46139d8ceff3da67951f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/f148abd99ca0429c826c6a37b9b5822c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/9a8b6b79f6444ef18ef8261ea25c9967, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/b17c4e437a0a4a80a80e2413ce18222b] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp, totalSize=25.3 K 2023-03-10 18:55:55,928 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 642fdd75ecec4371a15395bd1f1d5ab1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474544009 2023-03-10 18:55:55,928 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 68e7a146039c46139d8ceff3da67951f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474545404 2023-03-10 18:55:55,928 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting f148abd99ca0429c826c6a37b9b5822c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474547190 2023-03-10 18:55:55,929 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 9a8b6b79f6444ef18ef8261ea25c9967, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474548978 2023-03-10 18:55:55,929 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting b17c4e437a0a4a80a80e2413ce18222b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:55:55,937 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 86d4ba0f8509f65063fb59feedda6774#family2#compaction#152 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:56,010 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/08e65e3f8bb54dd8a50897c15b2b8afa, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/6dc195ec4bbc478a8e9db69620eddbbb, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/d4aa15e6b24c4a338c9d3efca95d4e9a, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/796ab348fa294d2ba0a9fc7cc9d3d85f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/18c1e0d78c4943219d42a9462ffa31fc] to archive 2023-03-10 18:55:56,011 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:56,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/08e65e3f8bb54dd8a50897c15b2b8afa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/08e65e3f8bb54dd8a50897c15b2b8afa 2023-03-10 18:55:56,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/6dc195ec4bbc478a8e9db69620eddbbb to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/6dc195ec4bbc478a8e9db69620eddbbb 2023-03-10 18:55:56,014 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/d4aa15e6b24c4a338c9d3efca95d4e9a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/d4aa15e6b24c4a338c9d3efca95d4e9a 2023-03-10 18:55:56,015 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/796ab348fa294d2ba0a9fc7cc9d3d85f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/796ab348fa294d2ba0a9fc7cc9d3d85f 2023-03-10 18:55:56,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/18c1e0d78c4943219d42a9462ffa31fc to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/18c1e0d78c4943219d42a9462ffa31fc 2023-03-10 18:55:56,345 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230310f1d975ef03474b9585d19899f3282a74_del 2023-03-10 18:55:56,345 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310f1d975ef03474b9585d19899f3282a74_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230310f1d975ef03474b9585d19899f3282a74_del 2023-03-10 18:55:56,880 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:55:56,880 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x236a9481 to 127.0.0.1:56431 2023-03-10 18:55:56,880 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:55:56,881 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:55:56,881 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b3a96c8 to 127.0.0.1:56431 2023-03-10 18:55:56,881 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:55:57,158 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d04f8346cd9480d9375d83bc3ec96e4 2023-03-10 18:55:57,159 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/.tmp/family2/9d04f8346cd9480d9375d83bc3ec96e4 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/9d04f8346cd9480d9375d83bc3ec96e4 2023-03-10 18:55:57,161 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d04f8346cd9480d9375d83bc3ec96e4 2023-03-10 18:55:57,162 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 86d4ba0f8509f65063fb59feedda6774/family2 of 86d4ba0f8509f65063fb59feedda6774 into 9d04f8346cd9480d9375d83bc3ec96e4(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:57,163 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 86d4ba0f8509f65063fb59feedda6774: 2023-03-10 18:55:57,163 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:57,163 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 3e4f64f1240b6c6abb2116376732f633/family1 is initiating major compaction (all files) 2023-03-10 18:55:57,163 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:57,163 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:57,164 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 3e4f64f1240b6c6abb2116376732f633/family1 in ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:57,164 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/1cb77d2862524774b9a862c42385a26f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/cd590da4656f4d5db26c8716612ff7ac, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/326e58cf1e7e4ff6a1bdc40ce22034b7, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/5c6b23f4a6a54e8dadcd9d4f598fa693, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/b110ef4291d74bc1b91142182f664168] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp, totalSize=26.6 K 2023-03-10 18:55:57,164 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 1cb77d2862524774b9a862c42385a26f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474544008 2023-03-10 18:55:57,165 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting cd590da4656f4d5db26c8716612ff7ac, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474545404 2023-03-10 18:55:57,165 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 326e58cf1e7e4ff6a1bdc40ce22034b7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474547190 2023-03-10 18:55:57,166 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 5c6b23f4a6a54e8dadcd9d4f598fa693, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474548978 2023-03-10 18:55:57,166 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting b110ef4291d74bc1b91142182f664168, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:55:57,174 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 3e4f64f1240b6c6abb2116376732f633#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:57,210 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/642fdd75ecec4371a15395bd1f1d5ab1, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/68e7a146039c46139d8ceff3da67951f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/f148abd99ca0429c826c6a37b9b5822c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/9a8b6b79f6444ef18ef8261ea25c9967, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/b17c4e437a0a4a80a80e2413ce18222b] to archive 2023-03-10 18:55:57,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:57,212 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/642fdd75ecec4371a15395bd1f1d5ab1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/642fdd75ecec4371a15395bd1f1d5ab1 2023-03-10 18:55:57,213 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/68e7a146039c46139d8ceff3da67951f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/68e7a146039c46139d8ceff3da67951f 2023-03-10 18:55:57,214 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/f148abd99ca0429c826c6a37b9b5822c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/f148abd99ca0429c826c6a37b9b5822c 2023-03-10 18:55:57,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/9a8b6b79f6444ef18ef8261ea25c9967 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/9a8b6b79f6444ef18ef8261ea25c9967 2023-03-10 18:55:57,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/b17c4e437a0a4a80a80e2413ce18222b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family2/b17c4e437a0a4a80a80e2413ce18222b 2023-03-10 18:55:57,582 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303106e3876daa8b1482caa2bdaa40be1139c_del 2023-03-10 18:55:57,582 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303106e3876daa8b1482caa2bdaa40be1139c_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303106e3876daa8b1482caa2bdaa40be1139c_del 2023-03-10 18:55:58,396 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 45cdd1809e7e4b599235ab47e14dc9ba 2023-03-10 18:55:58,396 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family1/45cdd1809e7e4b599235ab47e14dc9ba as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/45cdd1809e7e4b599235ab47e14dc9ba 2023-03-10 18:55:58,399 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 45cdd1809e7e4b599235ab47e14dc9ba 2023-03-10 18:55:58,400 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3e4f64f1240b6c6abb2116376732f633/family1 of 3e4f64f1240b6c6abb2116376732f633 into 45cdd1809e7e4b599235ab47e14dc9ba(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:58,400 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 3e4f64f1240b6c6abb2116376732f633: 2023-03-10 18:55:58,400 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:55:58,400 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 3e4f64f1240b6c6abb2116376732f633/family2 is initiating major compaction (all files) 2023-03-10 18:55:58,400 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:55:58,401 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:55:58,401 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 3e4f64f1240b6c6abb2116376732f633/family2 in ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:55:58,401 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/1efc4e36b0d34779a1e4990ad42d4f25, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/553bb13ab0754db49e803799daca7c4d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/34ee820edbbf487c8286ee1b2c11d314, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/f20ce428f7d647f2afa08e4a621c3e22, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/923d54907f8d4b1e8b1b6c87103c942c] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp, totalSize=25.3 K 2023-03-10 18:55:58,401 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 1efc4e36b0d34779a1e4990ad42d4f25, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474544008 2023-03-10 18:55:58,401 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 553bb13ab0754db49e803799daca7c4d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474545404 2023-03-10 18:55:58,402 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 34ee820edbbf487c8286ee1b2c11d314, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474547190 2023-03-10 18:55:58,402 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting f20ce428f7d647f2afa08e4a621c3e22, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474548978 2023-03-10 18:55:58,403 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 923d54907f8d4b1e8b1b6c87103c942c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:55:58,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/1cb77d2862524774b9a862c42385a26f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/cd590da4656f4d5db26c8716612ff7ac, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/326e58cf1e7e4ff6a1bdc40ce22034b7, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/5c6b23f4a6a54e8dadcd9d4f598fa693, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/b110ef4291d74bc1b91142182f664168] to archive 2023-03-10 18:55:58,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:58,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/1cb77d2862524774b9a862c42385a26f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/1cb77d2862524774b9a862c42385a26f 2023-03-10 18:55:58,414 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/cd590da4656f4d5db26c8716612ff7ac to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/cd590da4656f4d5db26c8716612ff7ac 2023-03-10 18:55:58,415 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/326e58cf1e7e4ff6a1bdc40ce22034b7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/326e58cf1e7e4ff6a1bdc40ce22034b7 2023-03-10 18:55:58,416 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/5c6b23f4a6a54e8dadcd9d4f598fa693 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/5c6b23f4a6a54e8dadcd9d4f598fa693 2023-03-10 18:55:58,417 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/b110ef4291d74bc1b91142182f664168 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/b110ef4291d74bc1b91142182f664168 2023-03-10 18:55:58,420 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 3e4f64f1240b6c6abb2116376732f633#family2#compaction#154 average throughput is 0.11 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:55:58,831 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230310ceb0f8783fee4332b12c536a52a91e29_del 2023-03-10 18:55:58,831 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310ceb0f8783fee4332b12c536a52a91e29_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230310ceb0f8783fee4332b12c536a52a91e29_del 2023-03-10 18:55:59,645 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e3773d4993c546fca80d97bd20c5d6d7 2023-03-10 18:55:59,645 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/.tmp/family2/e3773d4993c546fca80d97bd20c5d6d7 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/e3773d4993c546fca80d97bd20c5d6d7 2023-03-10 18:55:59,648 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e3773d4993c546fca80d97bd20c5d6d7 2023-03-10 18:55:59,649 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3e4f64f1240b6c6abb2116376732f633/family2 of 3e4f64f1240b6c6abb2116376732f633 into e3773d4993c546fca80d97bd20c5d6d7(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:55:59,649 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 3e4f64f1240b6c6abb2116376732f633: 2023-03-10 18:55:59,652 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:59,655 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:59,658 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:55:59,658 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:55:59,663 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-10 18:55:59,677 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-10 18:55:59,680 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303107a154ca1b8f44c1bb63c2b3d32ee8c87_del 2023-03-10 18:55:59,683 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303106e3876daa8b1482caa2bdaa40be1139c_del 2023-03-10 18:55:59,686 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230310b1a3efa9898f4fd2b769cbd003846bca_del 2023-03-10 18:55:59,686 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-10 18:55:59,686 DEBUG [Listener at localhost/36413] zookeeper.ReadOnlyZKClient(139): Connect 0x6da107a7 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:55:59,691 DEBUG [Listener at localhost/36413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e421598, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:55:59,692 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230310 for table='ns:testMinorCompaction' and column='family1' 2023-03-10 18:55:59,692 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230310 for table='ns:testMinorCompaction' and column='family1' 2023-03-10 18:55:59,692 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230310 for table='ns:testMinorCompaction' and column='family1' 2023-03-10 18:55:59,710 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/1efc4e36b0d34779a1e4990ad42d4f25, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/553bb13ab0754db49e803799daca7c4d, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/34ee820edbbf487c8286ee1b2c11d314, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/f20ce428f7d647f2afa08e4a621c3e22, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/923d54907f8d4b1e8b1b6c87103c942c] to archive 2023-03-10 18:55:59,711 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:59,712 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/1efc4e36b0d34779a1e4990ad42d4f25 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/1efc4e36b0d34779a1e4990ad42d4f25 2023-03-10 18:55:59,713 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/553bb13ab0754db49e803799daca7c4d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/553bb13ab0754db49e803799daca7c4d 2023-03-10 18:55:59,715 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/34ee820edbbf487c8286ee1b2c11d314 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/34ee820edbbf487c8286ee1b2c11d314 2023-03-10 18:55:59,716 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/f20ce428f7d647f2afa08e4a621c3e22 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/f20ce428f7d647f2afa08e4a621c3e22 2023-03-10 18:55:59,717 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/923d54907f8d4b1e8b1b6c87103c942c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family2/923d54907f8d4b1e8b1b6c87103c942c 2023-03-10 18:55:59,720 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303102973f6f97e394e369b85d94b8f7374a8 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303102973f6f97e394e369b85d94b8f7374a8 2023-03-10 18:55:59,721 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230310 table='ns:testMinorCompaction' column='family1' 2023-03-10 18:55:59,734 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:55:59,736 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43762, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:55:59,743 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:55:59,746 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49430, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:55:59,755 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230310/family1/89b68050b35144cd94970d82e076f423 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-10 18:55:59,756 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230310/family1/89b68050b35144cd94970d82e076f423}] 2023-03-10 18:55:59,760 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230310/family1/89b68050b35144cd94970d82e076f423 for inclusion in 86d4ba0f8509f65063fb59feedda6774/family1 2023-03-10 18:55:59,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-10 18:55:59,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-10 18:55:59,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 86d4ba0f8509f65063fb59feedda6774: 2023-03-10 18:55:59,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230310/family1/89b68050b35144cd94970d82e076f423 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__vo7tcgqcjci12all8asmuj1vlp27m44nougd5pcn6ve6a4j1teirqf19aig5smmp/family1/89b68050b35144cd94970d82e076f423 2023-03-10 18:55:59,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__vo7tcgqcjci12all8asmuj1vlp27m44nougd5pcn6ve6a4j1teirqf19aig5smmp/family1/89b68050b35144cd94970d82e076f423 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/36f6b52e15b94985803e76206f495522_SeqId_28_ 2023-03-10 18:55:59,765 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__vo7tcgqcjci12all8asmuj1vlp27m44nougd5pcn6ve6a4j1teirqf19aig5smmp/family1/89b68050b35144cd94970d82e076f423 into 86d4ba0f8509f65063fb59feedda6774/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/36f6b52e15b94985803e76206f495522_SeqId_28_ - updating store file list. 2023-03-10 18:55:59,768 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/36f6b52e15b94985803e76206f495522_SeqId_28_ into 86d4ba0f8509f65063fb59feedda6774/family1 2023-03-10 18:55:59,768 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__vo7tcgqcjci12all8asmuj1vlp27m44nougd5pcn6ve6a4j1teirqf19aig5smmp/family1/89b68050b35144cd94970d82e076f423 into 86d4ba0f8509f65063fb59feedda6774/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/family1/36f6b52e15b94985803e76206f495522_SeqId_28_) 2023-03-10 18:55:59,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__vo7tcgqcjci12all8asmuj1vlp27m44nougd5pcn6ve6a4j1teirqf19aig5smmp/family1/89b68050b35144cd94970d82e076f423 2023-03-10 18:55:59,770 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230310 table='ns:testMinorCompaction' column='family1' 2023-03-10 18:55:59,771 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:55:59,772 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230310366b079de2e9418aa4c02a3a389c1db9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230310366b079de2e9418aa4c02a3a389c1db9 2023-03-10 18:55:59,772 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-10 18:55:59,772 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230310' 2023-03-10 18:55:59,772 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230310 for table='ns:testMinorCompaction' and column='family1' 2023-03-10 18:56:00,117 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310b9cb0b2b576145e68bdfc3a3008043be to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230310b9cb0b2b576145e68bdfc3a3008043be 2023-03-10 18:56:00,118 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230310 table='ns:testMinorCompaction' column='family1' 2023-03-10 18:56:00,140 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230310/family1/81db3859c5f04ebea127a5f7a5823a3c first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-10 18:56:00,141 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230310/family1/81db3859c5f04ebea127a5f7a5823a3c}] 2023-03-10 18:56:00,142 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230310/family1/81db3859c5f04ebea127a5f7a5823a3c for inclusion in 6cce5e62535147bca4bf736faaf0f227/family1 2023-03-10 18:56:00,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-10 18:56:00,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last=1 2023-03-10 18:56:00,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 6cce5e62535147bca4bf736faaf0f227: 2023-03-10 18:56:00,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230310/family1/81db3859c5f04ebea127a5f7a5823a3c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__532lpbvln3a3aaeuo70kuars9iujh76kl8tpjggnlsr8sjaqqsbi3inafatgod2j/family1/81db3859c5f04ebea127a5f7a5823a3c 2023-03-10 18:56:00,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__532lpbvln3a3aaeuo70kuars9iujh76kl8tpjggnlsr8sjaqqsbi3inafatgod2j/family1/81db3859c5f04ebea127a5f7a5823a3c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/f6ccd2bfe86e4395979b46a8be531a5d_SeqId_28_ 2023-03-10 18:56:00,147 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__532lpbvln3a3aaeuo70kuars9iujh76kl8tpjggnlsr8sjaqqsbi3inafatgod2j/family1/81db3859c5f04ebea127a5f7a5823a3c into 6cce5e62535147bca4bf736faaf0f227/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/f6ccd2bfe86e4395979b46a8be531a5d_SeqId_28_ - updating store file list. 2023-03-10 18:56:00,149 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/f6ccd2bfe86e4395979b46a8be531a5d_SeqId_28_ into 6cce5e62535147bca4bf736faaf0f227/family1 2023-03-10 18:56:00,150 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__532lpbvln3a3aaeuo70kuars9iujh76kl8tpjggnlsr8sjaqqsbi3inafatgod2j/family1/81db3859c5f04ebea127a5f7a5823a3c into 6cce5e62535147bca4bf736faaf0f227/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/family1/f6ccd2bfe86e4395979b46a8be531a5d_SeqId_28_) 2023-03-10 18:56:00,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__532lpbvln3a3aaeuo70kuars9iujh76kl8tpjggnlsr8sjaqqsbi3inafatgod2j/family1/81db3859c5f04ebea127a5f7a5823a3c 2023-03-10 18:56:00,151 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230310 table='ns:testMinorCompaction' column='family1' 2023-03-10 18:56:00,152 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:00,154 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303109837794828184f59b764b721b1aaf7f7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303109837794828184f59b764b721b1aaf7f7 2023-03-10 18:56:00,154 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-10 18:56:00,154 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230310' 2023-03-10 18:56:00,154 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230310 for table='ns:testMinorCompaction' and column='family1' 2023-03-10 18:56:00,517 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310c8c7b6c5c1bd42538e0bdffc863d0ae9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230310c8c7b6c5c1bd42538e0bdffc863d0ae9 2023-03-10 18:56:00,517 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230310 table='ns:testMinorCompaction' column='family1' 2023-03-10 18:56:00,546 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230310/family1/bec494ac87f143e9a99e43f9c3d98d0d first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-10 18:56:00,547 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230310/family1/bec494ac87f143e9a99e43f9c3d98d0d}] 2023-03-10 18:56:00,548 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230310/family1/bec494ac87f143e9a99e43f9c3d98d0d for inclusion in 3e4f64f1240b6c6abb2116376732f633/family1 2023-03-10 18:56:00,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-10 18:56:00,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-10 18:56:00,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 3e4f64f1240b6c6abb2116376732f633: 2023-03-10 18:56:00,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230310/family1/bec494ac87f143e9a99e43f9c3d98d0d to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__jpd7halc0ort1adkmjdif6t529t0ug8m0q5v3tl6k1123svq3bhud8i09hujm941/family1/bec494ac87f143e9a99e43f9c3d98d0d 2023-03-10 18:56:00,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__jpd7halc0ort1adkmjdif6t529t0ug8m0q5v3tl6k1123svq3bhud8i09hujm941/family1/bec494ac87f143e9a99e43f9c3d98d0d as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/de40093cb1654872b179717fa6b7a796_SeqId_28_ 2023-03-10 18:56:00,553 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__jpd7halc0ort1adkmjdif6t529t0ug8m0q5v3tl6k1123svq3bhud8i09hujm941/family1/bec494ac87f143e9a99e43f9c3d98d0d into 3e4f64f1240b6c6abb2116376732f633/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/de40093cb1654872b179717fa6b7a796_SeqId_28_ - updating store file list. 2023-03-10 18:56:00,556 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/de40093cb1654872b179717fa6b7a796_SeqId_28_ into 3e4f64f1240b6c6abb2116376732f633/family1 2023-03-10 18:56:00,556 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__jpd7halc0ort1adkmjdif6t529t0ug8m0q5v3tl6k1123svq3bhud8i09hujm941/family1/bec494ac87f143e9a99e43f9c3d98d0d into 3e4f64f1240b6c6abb2116376732f633/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/family1/de40093cb1654872b179717fa6b7a796_SeqId_28_) 2023-03-10 18:56:00,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__ns_testMinorCompaction__jpd7halc0ort1adkmjdif6t529t0ug8m0q5v3tl6k1123svq3bhud8i09hujm941/family1/bec494ac87f143e9a99e43f9c3d98d0d 2023-03-10 18:56:00,558 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230310 table='ns:testMinorCompaction' column='family1' 2023-03-10 18:56:00,558 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:00,559 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303103158be288d7a481399aa6f4bfb880cfa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303103158be288d7a481399aa6f4bfb880cfa 2023-03-10 18:56:00,559 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-10 18:56:00,559 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230310' 2023-03-10 18:56:00,560 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230310 for table='ns:testMinorCompaction' and column='family1' 2023-03-10 18:56:00,560 INFO [Listener at localhost/36413] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-10 18:56:00,562 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:56:00,564 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:56:00,566 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:56:00,567 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:56:00,586 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=248 (was 284), OpenFileDescriptor=626 (was 758), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=172 (was 189), ProcessCount=170 (was 170), AvailableMemoryMB=5422 (was 5431) 2023-03-10 18:56:00,600 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=626, MaxFileDescriptor=60000, SystemLoadAverage=172, ProcessCount=170, AvailableMemoryMB=5421 2023-03-10 18:56:00,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-10 18:56:00,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:00,604 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:56:00,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-10 18:56:00,604 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:56:00,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-10 18:56:00,605 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd 2023-03-10 18:56:00,606 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd empty. 2023-03-10 18:56:00,606 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd 2023-03-10 18:56:00,606 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-10 18:56:00,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-10 18:56:00,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-10 18:56:01,017 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-10 18:56:01,018 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => d3d50fac089741f712af64c9ad1303fd, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:56:01,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-10 18:56:01,423 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:56:01,423 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing d3d50fac089741f712af64c9ad1303fd, disabling compactions & flushes 2023-03-10 18:56:01,423 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:01,423 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:01,423 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. after waiting 0 ms 2023-03-10 18:56:01,423 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:01,423 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:01,423 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:01,425 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:56:01,426 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678474561426"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474561426"}]},"ts":"1678474561426"} 2023-03-10 18:56:01,428 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-10 18:56:01,428 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:56:01,429 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474561428"}]},"ts":"1678474561428"} 2023-03-10 18:56:01,429 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-10 18:56:01,432 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d3d50fac089741f712af64c9ad1303fd, ASSIGN}] 2023-03-10 18:56:01,434 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d3d50fac089741f712af64c9ad1303fd, ASSIGN 2023-03-10 18:56:01,434 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d3d50fac089741f712af64c9ad1303fd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:56:01,585 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=d3d50fac089741f712af64c9ad1303fd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:01,585 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678474561585"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474561585"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474561585"}]},"ts":"1678474561585"} 2023-03-10 18:56:01,587 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure d3d50fac089741f712af64c9ad1303fd, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:56:01,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-10 18:56:01,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:01,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d3d50fac089741f712af64c9ad1303fd, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd.', STARTKEY => '', ENDKEY => ''} 2023-03-10 18:56:01,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy d3d50fac089741f712af64c9ad1303fd 2023-03-10 18:56:01,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:56:01,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d3d50fac089741f712af64c9ad1303fd 2023-03-10 18:56:01,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d3d50fac089741f712af64c9ad1303fd 2023-03-10 18:56:01,742 INFO [StoreOpener-d3d50fac089741f712af64c9ad1303fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d3d50fac089741f712af64c9ad1303fd 2023-03-10 18:56:01,743 DEBUG [StoreOpener-d3d50fac089741f712af64c9ad1303fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1 2023-03-10 18:56:01,743 DEBUG [StoreOpener-d3d50fac089741f712af64c9ad1303fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1 2023-03-10 18:56:01,744 INFO [StoreOpener-d3d50fac089741f712af64c9ad1303fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d3d50fac089741f712af64c9ad1303fd columnFamilyName family1 2023-03-10 18:56:01,745 INFO [StoreOpener-d3d50fac089741f712af64c9ad1303fd-1] regionserver.HStore(310): Store=d3d50fac089741f712af64c9ad1303fd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:56:01,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd 2023-03-10 18:56:01,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd 2023-03-10 18:56:01,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d3d50fac089741f712af64c9ad1303fd 2023-03-10 18:56:01,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:56:01,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d3d50fac089741f712af64c9ad1303fd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63050425, jitterRate=-0.06047545373439789}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-10 18:56:01,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:01,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd., pid=114, masterSystemTime=1678474561738 2023-03-10 18:56:01,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:01,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:01,752 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=d3d50fac089741f712af64c9ad1303fd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:01,752 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678474561752"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474561752"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474561752"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474561752"}]},"ts":"1678474561752"} 2023-03-10 18:56:01,754 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-10 18:56:01,754 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure d3d50fac089741f712af64c9ad1303fd, server=jenkins-hbase4.apache.org,43181,1678474414959 in 166 msec 2023-03-10 18:56:01,755 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-10 18:56:01,755 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d3d50fac089741f712af64c9ad1303fd, ASSIGN in 322 msec 2023-03-10 18:56:01,756 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:56:01,756 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474561756"}]},"ts":"1678474561756"} 2023-03-10 18:56:01,757 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-10 18:56:01,759 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:56:01,760 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1570 sec 2023-03-10 18:56:02,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-10 18:56:02,710 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-10 18:56:02,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:02,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,728 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:02,728 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:02,728 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:56:02,728 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:56:02,728 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,728 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,729 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,729 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:02,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:02,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:02,730 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,730 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:02,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,731 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,731 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,731 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:56:02,731 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:02,731 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:56:02,731 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:56:02,731 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:02,731 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:02,731 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. started... 2023-03-10 18:56:02,732 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3d50fac089741f712af64c9ad1303fd 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-10 18:56:02,752 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511307e60f2213bd3437581d9bf5c8b2ade51 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511307e60f2213bd3437581d9bf5c8b2ade51 2023-03-10 18:56:02,755 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/22263f5dbf5649ef8c397834f5e7ccaa 2023-03-10 18:56:02,759 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/22263f5dbf5649ef8c397834f5e7ccaa as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/22263f5dbf5649ef8c397834f5e7ccaa 2023-03-10 18:56:02,762 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/22263f5dbf5649ef8c397834f5e7ccaa, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-10 18:56:02,763 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for d3d50fac089741f712af64c9ad1303fd in 31ms, sequenceid=5, compaction requested=false 2023-03-10 18:56:02,763 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:02,764 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:02,764 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:02,764 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:56:02,764 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:56:02,764 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,764 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:56:02,764 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:56:02,766 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,766 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:02,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:02,766 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,766 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:02,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:02,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:02,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:02,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:56:02,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@360aa6b6[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:02,767 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:56:02,767 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,769 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,769 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:56:02,769 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:56:02,769 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:02,769 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,769 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:02,771 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:56:02,771 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,771 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:02,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:02,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:02,771 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:56:02,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:02,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:02,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:02,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:02,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:56:02,773 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:56:02,773 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:02,773 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:02,773 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,773 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:02,775 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,775 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:02,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,775 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:02,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:02,775 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:02,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:02,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:02,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:02,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,776 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:02,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,779 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,779 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:02,779 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:02,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:02,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:02,779 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:02,779 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:02,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:02,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:56:02,779 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,779 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,779 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:02,779 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:02,779 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:02,779 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:02,780 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,879 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:02,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:02,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:02,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:02,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,885 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:02,885 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:02,885 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:56:02,885 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:56:02,886 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,886 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,887 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,887 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:02,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:02,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:02,888 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:02,888 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:02,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:02,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:56:02,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:02,889 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:56:02,889 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:02,889 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:56:02,889 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:56:02,889 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:02,889 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:02,889 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. started... 2023-03-10 18:56:02,889 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3d50fac089741f712af64c9ad1303fd 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-10 18:56:03,299 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ebdb465896a8414ca0ef9bd46d73c007 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128ebdb465896a8414ca0ef9bd46d73c007 2023-03-10 18:56:03,704 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/d8f70aacab3b40f7a4be83f7ad11830e 2023-03-10 18:56:03,708 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/d8f70aacab3b40f7a4be83f7ad11830e as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/d8f70aacab3b40f7a4be83f7ad11830e 2023-03-10 18:56:03,711 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/d8f70aacab3b40f7a4be83f7ad11830e, entries=1, sequenceid=9, filesize=5.0 K 2023-03-10 18:56:03,712 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for d3d50fac089741f712af64c9ad1303fd in 823ms, sequenceid=9, compaction requested=false 2023-03-10 18:56:03,713 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:03,713 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:03,713 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:56:03,713 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:56:03,713 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,713 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:56:03,713 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:56:03,722 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,722 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:03,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:03,722 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,722 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:03,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:03,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:03,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:03,723 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:56:03,723 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b0716da[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:03,723 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:56:03,723 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,726 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,726 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:56:03,726 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:56:03,726 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:03,726 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,726 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:03,728 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:56:03,728 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:03,728 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,728 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:56:03,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:03,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:03,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:03,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:03,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:03,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:03,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:56:03,730 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:56:03,730 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:03,730 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:03,730 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,730 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:03,731 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:03,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:03,731 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,731 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:03,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:03,732 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:03,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:03,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:03,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,733 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:03,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,736 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,736 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:03,736 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,736 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:03,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:03,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:03,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:03,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:03,736 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:03,736 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,736 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,736 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,736 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:03,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:03,737 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:03,736 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:03,737 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,837 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:03,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:03,841 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:03,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:03,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,842 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:03,842 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:03,842 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:56:03,842 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:56:03,843 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,843 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,846 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,846 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:03,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:03,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:03,847 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:03,847 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:03,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,847 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:03,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,847 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,848 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:56:03,848 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:03,848 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:56:03,848 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:03,848 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:56:03,848 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:56:03,848 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:03,848 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:03,848 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. started... 2023-03-10 18:56:03,848 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3d50fac089741f712af64c9ad1303fd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:56:04,259 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120115ebb43186bb41b783ada2aa8fccb234 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120115ebb43186bb41b783ada2aa8fccb234 2023-03-10 18:56:04,664 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/7d6fd3fdedd941adb96c1f22d1c58b45 2023-03-10 18:56:04,669 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/7d6fd3fdedd941adb96c1f22d1c58b45 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7d6fd3fdedd941adb96c1f22d1c58b45 2023-03-10 18:56:04,673 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7d6fd3fdedd941adb96c1f22d1c58b45, entries=1, sequenceid=13, filesize=4.9 K 2023-03-10 18:56:04,673 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3d50fac089741f712af64c9ad1303fd in 825ms, sequenceid=13, compaction requested=true 2023-03-10 18:56:04,673 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:04,673 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:04,674 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:56:04,674 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:56:04,674 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,674 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:56:04,674 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:56:04,675 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,675 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:04,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:04,675 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,675 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:04,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:04,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:04,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:04,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:56:04,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@191a05ca[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:04,677 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:56:04,677 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,678 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,678 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,678 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:56:04,678 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:56:04,678 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:04,678 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:04,680 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:56:04,680 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,680 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:04,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:04,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:04,680 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:56:04,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:04,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:04,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:04,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:04,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:56:04,682 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:56:04,682 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:04,682 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:04,682 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,682 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:04,684 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,684 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,684 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:04,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:04,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:04,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:04,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:04,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:04,684 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:04,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,685 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:04,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,687 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,687 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:04,687 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,687 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:04,687 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:04,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:56:04,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:04,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:04,687 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:04,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:04,688 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,688 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,688 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,688 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,688 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:04,688 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:04,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:04,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:04,788 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:04,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:04,792 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:04,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:04,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,794 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:04,794 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:04,794 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:56:04,794 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:56:04,794 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,794 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,796 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,796 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:04,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:04,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:04,796 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:04,796 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:04,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,796 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:04,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,797 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,797 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:56:04,797 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:04,797 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:56:04,797 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:04,797 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:56:04,797 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:56:04,797 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:04,797 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:04,797 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. started... 2023-03-10 18:56:04,797 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3d50fac089741f712af64c9ad1303fd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:56:05,207 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f8280d9c2f8e47d0937824b88d20c314 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205f8280d9c2f8e47d0937824b88d20c314 2023-03-10 18:56:05,612 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/9e08afc6448945b9803966522ff65955 2023-03-10 18:56:05,615 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/9e08afc6448945b9803966522ff65955 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/9e08afc6448945b9803966522ff65955 2023-03-10 18:56:05,619 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/9e08afc6448945b9803966522ff65955, entries=1, sequenceid=17, filesize=4.9 K 2023-03-10 18:56:05,620 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3d50fac089741f712af64c9ad1303fd in 823ms, sequenceid=17, compaction requested=true 2023-03-10 18:56:05,620 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:05,620 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:05,620 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:56:05,620 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:56:05,620 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,621 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:56:05,621 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:56:05,624 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,624 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:05,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:05,624 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,624 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:05,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:05,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:05,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:05,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:56:05,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@663d3a62[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:05,625 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:56:05,625 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,627 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,627 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:56:05,627 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:56:05,627 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:05,627 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,627 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:05,628 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,628 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:56:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:05,629 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:05,629 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:56:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:05,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:05,631 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:56:05,631 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:56:05,631 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:05,631 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:05,631 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,631 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:05,632 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,632 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:05,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,632 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:05,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:05,633 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:05,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:05,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:05,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:05,634 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:05,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,639 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,639 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:05,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:05,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:05,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:05,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:05,639 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:05,639 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:05,639 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:56:05,639 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,639 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:05,639 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:05,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:05,639 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:05,640 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,640 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,640 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,640 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,740 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:05,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:05,744 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:05,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:05,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,745 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:05,745 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:05,745 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:56:05,745 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:56:05,746 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,746 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,747 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:05,747 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:05,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:05,748 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:05,748 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:05,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:05,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,748 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,748 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:56:05,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:05,748 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:56:05,749 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:05,749 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:56:05,749 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:56:05,749 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:05,749 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:05,749 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. started... 2023-03-10 18:56:05,749 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3d50fac089741f712af64c9ad1303fd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:56:06,160 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228bebdd93a2cd5498ea6ffaf7d68242b55 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228bebdd93a2cd5498ea6ffaf7d68242b55 2023-03-10 18:56:06,566 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/7e46fead35b3467295a0521e7f5c5a93 2023-03-10 18:56:06,569 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/7e46fead35b3467295a0521e7f5c5a93 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7e46fead35b3467295a0521e7f5c5a93 2023-03-10 18:56:06,572 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7e46fead35b3467295a0521e7f5c5a93, entries=1, sequenceid=21, filesize=4.9 K 2023-03-10 18:56:06,573 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3d50fac089741f712af64c9ad1303fd in 824ms, sequenceid=21, compaction requested=true 2023-03-10 18:56:06,573 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:06,573 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:06,573 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:56:06,573 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:56:06,573 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,573 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:56:06,573 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:56:06,575 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,575 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:06,575 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,575 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:06,576 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:56:06,576 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e11f05[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:06,576 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:56:06,577 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,578 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,578 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:56:06,578 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:56:06,578 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:06,578 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,578 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:06,580 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:56:06,580 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:06,580 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,580 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:56:06,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:06,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:06,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:06,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:06,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:06,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:06,582 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:56:06,582 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:56:06,582 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:06,582 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:06,582 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,582 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:06,584 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,584 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,584 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:06,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:06,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:06,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,584 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:06,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:06,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:06,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,585 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:06,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,590 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,590 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:06,590 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:06,590 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:06,590 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:06,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:56:06,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:06,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:06,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:06,590 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,590 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,590 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,591 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,591 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,591 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:06,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:06,591 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:06,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:06,691 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:06,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:06,695 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:06,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:06,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,697 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:06,697 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:06,697 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:56:06,697 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:56:06,697 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,697 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,698 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,698 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:06,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:06,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:06,699 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:06,699 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:06,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,699 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:06,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,699 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,700 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:56:06,700 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:06,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:56:06,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:06,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:56:06,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:56:06,700 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:06,700 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:06,700 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. started... 2023-03-10 18:56:06,700 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3d50fac089741f712af64c9ad1303fd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:56:07,110 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231941a07d8051541a9a5892aac5eab4e5c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231941a07d8051541a9a5892aac5eab4e5c 2023-03-10 18:56:07,278 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:56:07,516 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/acf726180076403a900f885283bd7ff5 2023-03-10 18:56:07,520 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/acf726180076403a900f885283bd7ff5 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/acf726180076403a900f885283bd7ff5 2023-03-10 18:56:07,523 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/acf726180076403a900f885283bd7ff5, entries=1, sequenceid=25, filesize=4.9 K 2023-03-10 18:56:07,524 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3d50fac089741f712af64c9ad1303fd in 824ms, sequenceid=25, compaction requested=true 2023-03-10 18:56:07,525 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:07,525 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:07,525 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:56:07,525 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:56:07,525 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,525 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:56:07,525 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:56:07,526 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,526 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:07,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:07,527 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,527 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:07,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:07,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:07,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:07,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:56:07,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@267378a5[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:07,528 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:56:07,528 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,530 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,531 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:56:07,531 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:56:07,531 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,531 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:07,531 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:07,532 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,532 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:56:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:07,532 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:07,532 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:56:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:07,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:07,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:07,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:07,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:56:07,534 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:56:07,534 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:07,534 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:07,534 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,534 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:07,535 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,535 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:07,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:07,536 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:07,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:07,536 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:07,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:07,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:07,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,537 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:07,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,539 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,539 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:07,539 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:07,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:07,539 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:07,539 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:07,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:56:07,539 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:07,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:07,539 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:07,539 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,540 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:07,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:07,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:07,540 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,540 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,540 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,640 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:07,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:07,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:07,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:07,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,646 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:07,646 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:07,646 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:56:07,646 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:56:07,646 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,646 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,649 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,649 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:07,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:07,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:07,649 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:07,649 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:07,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:07,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,650 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,650 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:56:07,650 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:07,650 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:56:07,650 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:07,650 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:56:07,650 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:56:07,650 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:07,650 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:07,650 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. started... 2023-03-10 18:56:07,650 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3d50fac089741f712af64c9ad1303fd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:56:08,060 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101fe83ea3df51b4e6484450c9c8e35c4b1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101fe83ea3df51b4e6484450c9c8e35c4b1 2023-03-10 18:56:08,465 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/665848aa0dc24dd8924d02031b7cf0b1 2023-03-10 18:56:08,469 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/665848aa0dc24dd8924d02031b7cf0b1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/665848aa0dc24dd8924d02031b7cf0b1 2023-03-10 18:56:08,471 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/665848aa0dc24dd8924d02031b7cf0b1, entries=1, sequenceid=29, filesize=4.9 K 2023-03-10 18:56:08,472 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3d50fac089741f712af64c9ad1303fd in 822ms, sequenceid=29, compaction requested=true 2023-03-10 18:56:08,472 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:08,472 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:08,472 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:56:08,472 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:56:08,473 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,473 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:56:08,473 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:56:08,474 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,474 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:08,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:08,474 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,475 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:08,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:08,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:08,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:08,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:56:08,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f8e0df4[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:08,476 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:56:08,476 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,477 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:56:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:56:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:08,477 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,477 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:08,478 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:56:08,479 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,479 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:08,479 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:56:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:08,481 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:56:08,481 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:56:08,481 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:08,481 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:08,481 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,481 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:08,483 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,483 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:08,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,483 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:08,484 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,485 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:08,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,487 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:08,487 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:08,487 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:08,487 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:08,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:56:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:08,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:08,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:08,487 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,487 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,487 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,487 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,487 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:08,487 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:08,487 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:08,487 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,587 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:08,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:08,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:08,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:08,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,593 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:08,593 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:08,593 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:56:08,593 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:56:08,594 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,594 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,595 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,595 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:08,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:08,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:08,595 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:08,595 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:08,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:08,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,596 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,596 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:56:08,596 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:08,596 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:56:08,596 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:08,596 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:56:08,597 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:56:08,597 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:08,597 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:08,597 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. started... 2023-03-10 18:56:08,597 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3d50fac089741f712af64c9ad1303fd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:56:09,007 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035504837eb6c64084aecfaca31cc84ab0 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601035504837eb6c64084aecfaca31cc84ab0 2023-03-10 18:56:09,412 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/a99a2255eef0471cb072e2fd12035d24 2023-03-10 18:56:09,416 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/a99a2255eef0471cb072e2fd12035d24 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/a99a2255eef0471cb072e2fd12035d24 2023-03-10 18:56:09,419 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/a99a2255eef0471cb072e2fd12035d24, entries=1, sequenceid=33, filesize=4.9 K 2023-03-10 18:56:09,420 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3d50fac089741f712af64c9ad1303fd in 822ms, sequenceid=33, compaction requested=true 2023-03-10 18:56:09,420 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:09,420 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:09,420 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:56:09,420 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:56:09,420 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,420 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:56:09,420 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:56:09,422 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,422 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:09,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:09,422 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,422 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:09,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:09,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:09,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:09,423 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:56:09,423 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60efc44d[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:09,423 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:56:09,423 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,425 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,425 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:56:09,426 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:56:09,426 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,426 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:09,426 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:09,427 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:56:09,427 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,427 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:09,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:09,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:09,427 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:56:09,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:09,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:09,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:09,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:09,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:56:09,429 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:56:09,429 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:09,429 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:09,429 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,429 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:09,430 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,430 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,430 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:09,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:09,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:09,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:09,431 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,432 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,437 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,437 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,437 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:09,437 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:09,437 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,437 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,437 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:09,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:56:09,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:09,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:09,437 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:09,437 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:09,437 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,438 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:09,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:09,438 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:09,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:09,538 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:09,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:09,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:09,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:09,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,544 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:09,544 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:09,544 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:56:09,544 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:56:09,544 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,544 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,546 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,546 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:09,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:09,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:09,546 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,546 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:09,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,546 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:09,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,547 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,547 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:56:09,547 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,547 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:56:09,547 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:09,547 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:56:09,547 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:56:09,548 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:09,548 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:09,548 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. started... 2023-03-10 18:56:09,548 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3d50fac089741f712af64c9ad1303fd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:56:09,558 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907a70697bac4b74ec0a9e64fc7e9e775a4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907a70697bac4b74ec0a9e64fc7e9e775a4 2023-03-10 18:56:09,962 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/6555267e22794ebb806e587bfb332eea 2023-03-10 18:56:09,966 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/6555267e22794ebb806e587bfb332eea as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/6555267e22794ebb806e587bfb332eea 2023-03-10 18:56:09,970 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/6555267e22794ebb806e587bfb332eea, entries=1, sequenceid=37, filesize=4.9 K 2023-03-10 18:56:09,971 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3d50fac089741f712af64c9ad1303fd in 423ms, sequenceid=37, compaction requested=true 2023-03-10 18:56:09,971 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:09,971 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:09,971 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:56:09,971 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:56:09,971 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,971 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:56:09,971 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:56:09,973 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,973 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:09,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:09,973 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,973 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:09,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:09,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:09,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:09,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:56:09,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d7bf662[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:09,974 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:56:09,974 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,976 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,976 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:56:09,976 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:56:09,976 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,976 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:09,976 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:09,979 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:56:09,979 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:09,979 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,979 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:56:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:09,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:09,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:09,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:56:09,981 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:56:09,981 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:09,981 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:09,981 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,981 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:09,983 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,983 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:09,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:09,983 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:09,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:09,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:09,983 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:09,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:09,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,984 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:09,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,986 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,987 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,987 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:09,987 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:09,987 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:09,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:09,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:09,987 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:09,987 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:09,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:09,987 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,987 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,987 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:09,987 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:09,987 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:09,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:09,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:10,087 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:10,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:10,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:10,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:10,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,093 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:10,093 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:10,093 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-10 18:56:10,093 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-10 18:56:10,094 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,094 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,095 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,095 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:10,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:10,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:10,096 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,096 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:10,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,096 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:10,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,096 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,096 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-10 18:56:10,097 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,097 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-10 18:56:10,097 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:10,097 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-10 18:56:10,097 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-10 18:56:10,097 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:10,097 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:10,097 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. started... 2023-03-10 18:56:10,097 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3d50fac089741f712af64c9ad1303fd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-10 18:56:10,507 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120d56c04c8007340ecaf7e23dc7fce714f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120d56c04c8007340ecaf7e23dc7fce714f 2023-03-10 18:56:10,912 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/e68d774b68854ef8bf20ad6f7798966b 2023-03-10 18:56:10,916 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/e68d774b68854ef8bf20ad6f7798966b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e68d774b68854ef8bf20ad6f7798966b 2023-03-10 18:56:10,919 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e68d774b68854ef8bf20ad6f7798966b, entries=1, sequenceid=41, filesize=4.9 K 2023-03-10 18:56:10,919 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3d50fac089741f712af64c9ad1303fd in 822ms, sequenceid=41, compaction requested=true 2023-03-10 18:56:10,920 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:10,920 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:10,920 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-10 18:56:10,920 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-10 18:56:10,920 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,920 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-10 18:56:10,920 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-10 18:56:10,922 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,922 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:10,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:10,923 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,923 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:10,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:10,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:10,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:10,924 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-10 18:56:10,924 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50f01d38[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:10,924 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-10 18:56:10,924 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,925 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,925 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-10 18:56:10,925 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-10 18:56:10,925 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:10,925 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,925 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:10,926 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,926 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-10 18:56:10,926 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:10,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:10,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:10,927 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-10 18:56:10,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:10,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:10,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:10,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:10,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-10 18:56:10,929 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-10 18:56:10,929 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:10,929 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:10,929 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,929 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:10,930 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,930 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:10,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:10,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,930 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:10,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:10,931 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:10,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:10,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:10,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,932 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:10,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,935 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,935 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:10,935 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:10,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-10 18:56:10,935 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:10,935 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,935 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:10,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-10 18:56:10,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:10,935 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:10,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:10,936 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,936 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,936 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:10,936 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:10,936 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:10,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:10,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:11,036 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:11,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:11,038 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-10 18:56:11,039 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:56:11,040 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:56:11,041 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:56:11,041 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475171041 2023-03-10 18:56:11,041 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-10 18:56:11,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-10 18:56:11,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:56:11,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x650a6872 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:56:11,047 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@771e78be, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:56:11,047 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:56:11,871 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034c2ab3e39db24167a3b8af863bb19a21 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601034c2ab3e39db24167a3b8af863bb19a21 2023-03-10 18:56:11,872 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:56:11,884 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:56:11,886 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59760, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:56:11,890 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:56:11,891 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50500, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:56:11,899 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/eb38f865e63f420986af3dfc6038729c first=Optional[r5] last=Optional[r6] 2023-03-10 18:56:11,900 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/eb38f865e63f420986af3dfc6038729c}] 2023-03-10 18:56:11,904 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/eb38f865e63f420986af3dfc6038729c for inclusion in d3d50fac089741f712af64c9ad1303fd/family1 2023-03-10 18:56:11,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-10 18:56:11,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:56:11,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:11,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/eb38f865e63f420986af3dfc6038729c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lr0qn2u6uj03av1e80vd9ck3k8h4gvl6rfsqp6rqo1ibg3nvaoe80p2komuho2gp/family1/eb38f865e63f420986af3dfc6038729c 2023-03-10 18:56:11,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lr0qn2u6uj03av1e80vd9ck3k8h4gvl6rfsqp6rqo1ibg3nvaoe80p2komuho2gp/family1/eb38f865e63f420986af3dfc6038729c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e4818a02037844599c7c537ae3f5f32a_SeqId_44_ 2023-03-10 18:56:11,910 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lr0qn2u6uj03av1e80vd9ck3k8h4gvl6rfsqp6rqo1ibg3nvaoe80p2komuho2gp/family1/eb38f865e63f420986af3dfc6038729c into d3d50fac089741f712af64c9ad1303fd/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e4818a02037844599c7c537ae3f5f32a_SeqId_44_ - updating store file list. 2023-03-10 18:56:11,913 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e4818a02037844599c7c537ae3f5f32a_SeqId_44_ into d3d50fac089741f712af64c9ad1303fd/family1 2023-03-10 18:56:11,913 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lr0qn2u6uj03av1e80vd9ck3k8h4gvl6rfsqp6rqo1ibg3nvaoe80p2komuho2gp/family1/eb38f865e63f420986af3dfc6038729c into d3d50fac089741f712af64c9ad1303fd/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e4818a02037844599c7c537ae3f5f32a_SeqId_44_) 2023-03-10 18:56:11,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lr0qn2u6uj03av1e80vd9ck3k8h4gvl6rfsqp6rqo1ibg3nvaoe80p2komuho2gp/family1/eb38f865e63f420986af3dfc6038729c 2023-03-10 18:56:11,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-10 18:56:11,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-10 18:56:11,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(1912): d3d50fac089741f712af64c9ad1303fd/family1 is initiating minor compaction 2023-03-10 18:56:11,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6ae178c6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:56:11,916 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d3d50fac089741f712af64c9ad1303fd/family1 in testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:11,916 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/d8f70aacab3b40f7a4be83f7ad11830e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7d6fd3fdedd941adb96c1f22d1c58b45, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/9e08afc6448945b9803966522ff65955, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7e46fead35b3467295a0521e7f5c5a93, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/acf726180076403a900f885283bd7ff5, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/665848aa0dc24dd8924d02031b7cf0b1, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/a99a2255eef0471cb072e2fd12035d24, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/6555267e22794ebb806e587bfb332eea, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e68d774b68854ef8bf20ad6f7798966b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e4818a02037844599c7c537ae3f5f32a_SeqId_44_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp, totalSize=49.7 K 2023-03-10 18:56:11,916 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting d8f70aacab3b40f7a4be83f7ad11830e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-10 18:56:11,916 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 7d6fd3fdedd941adb96c1f22d1c58b45, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-10 18:56:11,916 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 9e08afc6448945b9803966522ff65955, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-10 18:56:11,916 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 7e46fead35b3467295a0521e7f5c5a93, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-10 18:56:11,916 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting acf726180076403a900f885283bd7ff5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-10 18:56:11,916 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 665848aa0dc24dd8924d02031b7cf0b1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-10 18:56:11,916 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting a99a2255eef0471cb072e2fd12035d24, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-10 18:56:11,916 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 6555267e22794ebb806e587bfb332eea, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-10 18:56:11,916 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting e68d774b68854ef8bf20ad6f7798966b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-10 18:56:11,916 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting e4818a02037844599c7c537ae3f5f32a_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-10 18:56:11,917 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:56:11,918 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:11,919 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101fe83ea3df51b4e6484450c9c8e35c4b1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101fe83ea3df51b4e6484450c9c8e35c4b1 2023-03-10 18:56:11,920 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601035504837eb6c64084aecfaca31cc84ab0 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601035504837eb6c64084aecfaca31cc84ab0 2023-03-10 18:56:11,920 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-10 18:56:11,920 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-10 18:56:11,921 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:56:11,921 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:56:11,936 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): d3d50fac089741f712af64c9ad1303fd#family1#compaction#165 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:56:12,748 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/a3fad24411e14c02b6a774092aab9432 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/a3fad24411e14c02b6a774092aab9432 2023-03-10 18:56:12,752 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in d3d50fac089741f712af64c9ad1303fd/family1 of d3d50fac089741f712af64c9ad1303fd into a3fad24411e14c02b6a774092aab9432(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:56:12,752 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:12,752 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678474571915; duration=0sec 2023-03-10 18:56:12,752 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:56:12,756 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316545ef6d709b48f59ac0c4ba8047b7d3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316545ef6d709b48f59ac0c4ba8047b7d3 2023-03-10 18:56:12,757 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:56:12,779 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ffaca03c0b99439883feedc05ab347b9 first=Optional[r1] last=Optional[r4] 2023-03-10 18:56:12,779 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ffaca03c0b99439883feedc05ab347b9}] 2023-03-10 18:56:12,781 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ffaca03c0b99439883feedc05ab347b9 for inclusion in d3d50fac089741f712af64c9ad1303fd/family1 2023-03-10 18:56:12,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-10 18:56:12,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:56:12,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:12,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ffaca03c0b99439883feedc05ab347b9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__t7ss2jtdk0l1bgfsc4ltv5bei12192ecicvj767dv4suf3fr3pj8gq86liruk74i/family1/ffaca03c0b99439883feedc05ab347b9 2023-03-10 18:56:12,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__t7ss2jtdk0l1bgfsc4ltv5bei12192ecicvj767dv4suf3fr3pj8gq86liruk74i/family1/ffaca03c0b99439883feedc05ab347b9 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/1b9311dfe5f740c4a859b6521c28b774_SeqId_47_ 2023-03-10 18:56:12,786 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__t7ss2jtdk0l1bgfsc4ltv5bei12192ecicvj767dv4suf3fr3pj8gq86liruk74i/family1/ffaca03c0b99439883feedc05ab347b9 into d3d50fac089741f712af64c9ad1303fd/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/1b9311dfe5f740c4a859b6521c28b774_SeqId_47_ - updating store file list. 2023-03-10 18:56:12,788 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/1b9311dfe5f740c4a859b6521c28b774_SeqId_47_ into d3d50fac089741f712af64c9ad1303fd/family1 2023-03-10 18:56:12,789 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__t7ss2jtdk0l1bgfsc4ltv5bei12192ecicvj767dv4suf3fr3pj8gq86liruk74i/family1/ffaca03c0b99439883feedc05ab347b9 into d3d50fac089741f712af64c9ad1303fd/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/1b9311dfe5f740c4a859b6521c28b774_SeqId_47_) 2023-03-10 18:56:12,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__t7ss2jtdk0l1bgfsc4ltv5bei12192ecicvj767dv4suf3fr3pj8gq86liruk74i/family1/ffaca03c0b99439883feedc05ab347b9 2023-03-10 18:56:12,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-10 18:56:12,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-10 18:56:12,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(1912): d3d50fac089741f712af64c9ad1303fd/family1 is initiating minor compaction (all files) 2023-03-10 18:56:12,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@564bf1f2; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:56:12,790 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d3d50fac089741f712af64c9ad1303fd/family1 in testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:12,790 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/22263f5dbf5649ef8c397834f5e7ccaa, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/a3fad24411e14c02b6a774092aab9432, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/1b9311dfe5f740c4a859b6521c28b774_SeqId_47_] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp, totalSize=178.1 K 2023-03-10 18:56:12,790 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 22263f5dbf5649ef8c397834f5e7ccaa, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839360 2023-03-10 18:56:12,791 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting a3fad24411e14c02b6a774092aab9432, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239360 2023-03-10 18:56:12,791 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] compactions.Compactor(207): Compacting 1b9311dfe5f740c4a859b6521c28b774_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965239360 2023-03-10 18:56:12,791 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:56:12,792 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:12,793 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120115ebb43186bb41b783ada2aa8fccb234 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120115ebb43186bb41b783ada2aa8fccb234 2023-03-10 18:56:12,794 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205f8280d9c2f8e47d0937824b88d20c314 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205f8280d9c2f8e47d0937824b88d20c314 2023-03-10 18:56:12,795 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228bebdd93a2cd5498ea6ffaf7d68242b55 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228bebdd93a2cd5498ea6ffaf7d68242b55 2023-03-10 18:56:12,796 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231941a07d8051541a9a5892aac5eab4e5c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231941a07d8051541a9a5892aac5eab4e5c 2023-03-10 18:56:12,797 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-10 18:56:12,797 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-10 18:56:12,797 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:56:12,797 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:56:12,806 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] throttle.PressureAwareThroughputController(145): d3d50fac089741f712af64c9ad1303fd#family1#compaction#166 average throughput is 17.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:56:12,811 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/d8f70aacab3b40f7a4be83f7ad11830e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7d6fd3fdedd941adb96c1f22d1c58b45, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/9e08afc6448945b9803966522ff65955, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7e46fead35b3467295a0521e7f5c5a93, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/acf726180076403a900f885283bd7ff5, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/665848aa0dc24dd8924d02031b7cf0b1, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/a99a2255eef0471cb072e2fd12035d24, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/6555267e22794ebb806e587bfb332eea, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e68d774b68854ef8bf20ad6f7798966b, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e4818a02037844599c7c537ae3f5f32a_SeqId_44_] to archive 2023-03-10 18:56:12,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:12,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/d8f70aacab3b40f7a4be83f7ad11830e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/d8f70aacab3b40f7a4be83f7ad11830e 2023-03-10 18:56:12,816 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7d6fd3fdedd941adb96c1f22d1c58b45 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7d6fd3fdedd941adb96c1f22d1c58b45 2023-03-10 18:56:12,817 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/9e08afc6448945b9803966522ff65955 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/9e08afc6448945b9803966522ff65955 2023-03-10 18:56:12,818 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7e46fead35b3467295a0521e7f5c5a93 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/7e46fead35b3467295a0521e7f5c5a93 2023-03-10 18:56:12,819 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/acf726180076403a900f885283bd7ff5 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/acf726180076403a900f885283bd7ff5 2023-03-10 18:56:12,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/665848aa0dc24dd8924d02031b7cf0b1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/665848aa0dc24dd8924d02031b7cf0b1 2023-03-10 18:56:12,821 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/a99a2255eef0471cb072e2fd12035d24 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/a99a2255eef0471cb072e2fd12035d24 2023-03-10 18:56:12,821 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/6555267e22794ebb806e587bfb332eea to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/6555267e22794ebb806e587bfb332eea 2023-03-10 18:56:12,822 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e68d774b68854ef8bf20ad6f7798966b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e68d774b68854ef8bf20ad6f7798966b 2023-03-10 18:56:12,824 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e4818a02037844599c7c537ae3f5f32a_SeqId_44_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/e4818a02037844599c7c537ae3f5f32a_SeqId_44_ 2023-03-10 18:56:12,840 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511302da6c775ddd64552b2952eae27ec36ef to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511302da6c775ddd64552b2952eae27ec36ef 2023-03-10 18:56:12,841 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:56:12,863 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4aca339f39f84bf4b72919710fa8afb1 first=Optional[r00] last=Optional[r8] 2023-03-10 18:56:12,863 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4aca339f39f84bf4b72919710fa8afb1}] 2023-03-10 18:56:12,865 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4aca339f39f84bf4b72919710fa8afb1 for inclusion in d3d50fac089741f712af64c9ad1303fd/family1 2023-03-10 18:56:12,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-10 18:56:12,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last= 2023-03-10 18:56:12,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:12,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4aca339f39f84bf4b72919710fa8afb1 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__momt6543sl6pqqp3cdqkvgnh3etcc92tn09g75ca5cr3ng87kg2g6f2g1q5obo39/family1/4aca339f39f84bf4b72919710fa8afb1 2023-03-10 18:56:12,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__momt6543sl6pqqp3cdqkvgnh3etcc92tn09g75ca5cr3ng87kg2g6f2g1q5obo39/family1/4aca339f39f84bf4b72919710fa8afb1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/5dae708b7e2f45a5bdbf9f6ee84881d2_SeqId_49_ 2023-03-10 18:56:12,871 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__momt6543sl6pqqp3cdqkvgnh3etcc92tn09g75ca5cr3ng87kg2g6f2g1q5obo39/family1/4aca339f39f84bf4b72919710fa8afb1 into d3d50fac089741f712af64c9ad1303fd/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/5dae708b7e2f45a5bdbf9f6ee84881d2_SeqId_49_ - updating store file list. 2023-03-10 18:56:12,874 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/5dae708b7e2f45a5bdbf9f6ee84881d2_SeqId_49_ into d3d50fac089741f712af64c9ad1303fd/family1 2023-03-10 18:56:12,874 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__momt6543sl6pqqp3cdqkvgnh3etcc92tn09g75ca5cr3ng87kg2g6f2g1q5obo39/family1/4aca339f39f84bf4b72919710fa8afb1 into d3d50fac089741f712af64c9ad1303fd/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/5dae708b7e2f45a5bdbf9f6ee84881d2_SeqId_49_) 2023-03-10 18:56:12,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMinorCompactionWithMonthlyPolicy__momt6543sl6pqqp3cdqkvgnh3etcc92tn09g75ca5cr3ng87kg2g6f2g1q5obo39/family1/4aca339f39f84bf4b72919710fa8afb1 2023-03-10 18:56:12,876 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:56:12,877 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:12,878 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120d56c04c8007340ecaf7e23dc7fce714f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120d56c04c8007340ecaf7e23dc7fce714f 2023-03-10 18:56:12,879 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128ebdb465896a8414ca0ef9bd46d73c007 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128ebdb465896a8414ca0ef9bd46d73c007 2023-03-10 18:56:12,880 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511307e60f2213bd3437581d9bf5c8b2ade51 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511307e60f2213bd3437581d9bf5c8b2ade51 2023-03-10 18:56:12,880 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-10 18:56:12,880 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-10 18:56:12,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:56:12,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:56:12,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-10 18:56:12,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-10 18:56:12,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:56:12,882 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-10 18:56:12,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.8440 sec 2023-03-10 18:56:13,091 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=261 (was 248) - Thread LEAK? -, OpenFileDescriptor=655 (was 626) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=161 (was 172), ProcessCount=170 (was 170), AvailableMemoryMB=5415 (was 5421) 2023-03-10 18:56:13,106 INFO [Listener at localhost/36413] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=261, OpenFileDescriptor=655, MaxFileDescriptor=60000, SystemLoadAverage=161, ProcessCount=170, AvailableMemoryMB=5414 2023-03-10 18:56:13,132 DEBUG [Listener at localhost/36413] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-10 18:56:13,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x1C\xB6\x90\xF7HG\x9D\x1At\xA5\xC6\xC5S\x17)/"\x10\x1F\xFD\xB3\x5C\xFE\x0C\xD9\x9B<1\x92B(sUu*\x10N@/v)\x82\xE6\xDD\x91,\x01\xEF\xAF*\x81\xF62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-10 18:56:13,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-10 18:56:13,202 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-10 18:56:13,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-10 18:56:13,203 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-10 18:56:13,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-10 18:56:13,205 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a 2023-03-10 18:56:13,205 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:13,206 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e 2023-03-10 18:56:13,206 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a empty. 2023-03-10 18:56:13,206 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a empty. 2023-03-10 18:56:13,206 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e empty. 2023-03-10 18:56:13,206 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a 2023-03-10 18:56:13,206 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:13,206 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e 2023-03-10 18:56:13,207 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-10 18:56:13,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-10 18:56:13,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-10 18:56:13,614 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-10 18:56:13,616 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 21cbab1dd7bfd57fa395887a9422884e, NAME => 'testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x1C\xB6\x90\xF7HG\x9D\x1At\xA5\xC6\xC5S\x17)/"\x10\x1F\xFD\xB3\x5C\xFE\x0C\xD9\x9B<1\x92B(sUu*\x10N@/v)\x82\xE6\xDD\x91,\x01\xEF\xAF*\x81\xF62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:56:13,616 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => 40054fa4a1cc8f5fa36fc70cc0b9a27a, NAME => 'testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x1C\xB6\x90\xF7HG\x9D\x1At\xA5\xC6\xC5S\x17)/"\x10\x1F\xFD\xB3\x5C\xFE\x0C\xD9\x9B<1\x92B(sUu*\x10N@/v)\x82\xE6\xDD\x91,\x01\xEF\xAF*\x81\xF62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:56:13,616 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => e86ad0a683833caf76b40c433c7efc0a, NAME => 'testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x1C\xB6\x90\xF7HG\x9D\x1At\xA5\xC6\xC5S\x17)/"\x10\x1F\xFD\xB3\x5C\xFE\x0C\xD9\x9B<1\x92B(sUu*\x10N@/v)\x82\xE6\xDD\x91,\x01\xEF\xAF*\x81\xF62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/.tmp 2023-03-10 18:56:13,627 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:56:13,627 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 21cbab1dd7bfd57fa395887a9422884e, disabling compactions & flushes 2023-03-10 18:56:13,627 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:13,627 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:13,627 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. after waiting 0 ms 2023-03-10 18:56:13,627 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:13,627 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:13,627 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 21cbab1dd7bfd57fa395887a9422884e: 2023-03-10 18:56:13,636 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/.tmp/family1/5aa33dac19454d30915f3cd9c1dccf63 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/5aa33dac19454d30915f3cd9c1dccf63 2023-03-10 18:56:13,640 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d3d50fac089741f712af64c9ad1303fd/family1 of d3d50fac089741f712af64c9ad1303fd into 5aa33dac19454d30915f3cd9c1dccf63(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-10 18:56:13,640 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:13,640 INFO [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678474572790; duration=0sec 2023-03-10 18:56:13,640 DEBUG [RS:0;jenkins-hbase4:43181-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-10 18:56:13,710 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/22263f5dbf5649ef8c397834f5e7ccaa, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/a3fad24411e14c02b6a774092aab9432, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/1b9311dfe5f740c4a859b6521c28b774_SeqId_47_] to archive 2023-03-10 18:56:13,711 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:13,712 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/22263f5dbf5649ef8c397834f5e7ccaa to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/22263f5dbf5649ef8c397834f5e7ccaa 2023-03-10 18:56:13,713 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/a3fad24411e14c02b6a774092aab9432 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/a3fad24411e14c02b6a774092aab9432 2023-03-10 18:56:13,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/1b9311dfe5f740c4a859b6521c28b774_SeqId_47_ to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/family1/1b9311dfe5f740c4a859b6521c28b774_SeqId_47_ 2023-03-10 18:56:13,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-10 18:56:14,027 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:56:14,027 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing 40054fa4a1cc8f5fa36fc70cc0b9a27a, disabling compactions & flushes 2023-03-10 18:56:14,027 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:14,027 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:14,027 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. after waiting 0 ms 2023-03-10 18:56:14,027 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:14,027 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:14,027 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for 40054fa4a1cc8f5fa36fc70cc0b9a27a: 2023-03-10 18:56:14,028 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:56:14,028 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing e86ad0a683833caf76b40c433c7efc0a, disabling compactions & flushes 2023-03-10 18:56:14,028 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:14,028 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:14,028 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. after waiting 0 ms 2023-03-10 18:56:14,028 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:14,028 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:14,028 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for e86ad0a683833caf76b40c433c7efc0a: 2023-03-10 18:56:14,030 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-10 18:56:14,030 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474574030"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474574030"}]},"ts":"1678474574030"} 2023-03-10 18:56:14,031 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474574030"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474574030"}]},"ts":"1678474574030"} 2023-03-10 18:56:14,031 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678474574030"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678474574030"}]},"ts":"1678474574030"} 2023-03-10 18:56:14,032 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-10 18:56:14,033 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-10 18:56:14,033 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474574033"}]},"ts":"1678474574033"} 2023-03-10 18:56:14,034 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-10 18:56:14,038 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=40054fa4a1cc8f5fa36fc70cc0b9a27a, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=e86ad0a683833caf76b40c433c7efc0a, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=21cbab1dd7bfd57fa395887a9422884e, ASSIGN}] 2023-03-10 18:56:14,039 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=21cbab1dd7bfd57fa395887a9422884e, ASSIGN 2023-03-10 18:56:14,039 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=e86ad0a683833caf76b40c433c7efc0a, ASSIGN 2023-03-10 18:56:14,039 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=40054fa4a1cc8f5fa36fc70cc0b9a27a, ASSIGN 2023-03-10 18:56:14,040 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=21cbab1dd7bfd57fa395887a9422884e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:56:14,040 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=40054fa4a1cc8f5fa36fc70cc0b9a27a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:56:14,040 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=e86ad0a683833caf76b40c433c7efc0a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43181,1678474414959; forceNewPlan=false, retain=false 2023-03-10 18:56:14,191 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=40054fa4a1cc8f5fa36fc70cc0b9a27a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:14,191 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=e86ad0a683833caf76b40c433c7efc0a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:14,191 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474574191"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474574191"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474574191"}]},"ts":"1678474574191"} 2023-03-10 18:56:14,191 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=21cbab1dd7bfd57fa395887a9422884e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:14,192 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678474574191"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474574191"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474574191"}]},"ts":"1678474574191"} 2023-03-10 18:56:14,192 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474574191"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678474574191"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678474574191"}]},"ts":"1678474574191"} 2023-03-10 18:56:14,193 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure 40054fa4a1cc8f5fa36fc70cc0b9a27a, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:56:14,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure e86ad0a683833caf76b40c433c7efc0a, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:56:14,194 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure 21cbab1dd7bfd57fa395887a9422884e, server=jenkins-hbase4.apache.org,43181,1678474414959}] 2023-03-10 18:56:14,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-10 18:56:14,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:14,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e86ad0a683833caf76b40c433c7efc0a, NAME => 'testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a.', STARTKEY => '1', ENDKEY => '2'} 2023-03-10 18:56:14,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:14,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:56:14,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:14,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:14,369 INFO [StoreOpener-e86ad0a683833caf76b40c433c7efc0a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:14,370 DEBUG [StoreOpener-e86ad0a683833caf76b40c433c7efc0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1 2023-03-10 18:56:14,370 DEBUG [StoreOpener-e86ad0a683833caf76b40c433c7efc0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1 2023-03-10 18:56:14,370 INFO [StoreOpener-e86ad0a683833caf76b40c433c7efc0a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e86ad0a683833caf76b40c433c7efc0a columnFamilyName family1 2023-03-10 18:56:14,371 INFO [StoreOpener-e86ad0a683833caf76b40c433c7efc0a-1] regionserver.HStore(310): Store=e86ad0a683833caf76b40c433c7efc0a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:56:14,372 INFO [StoreOpener-e86ad0a683833caf76b40c433c7efc0a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:14,373 DEBUG [StoreOpener-e86ad0a683833caf76b40c433c7efc0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2 2023-03-10 18:56:14,373 DEBUG [StoreOpener-e86ad0a683833caf76b40c433c7efc0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2 2023-03-10 18:56:14,373 INFO [StoreOpener-e86ad0a683833caf76b40c433c7efc0a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e86ad0a683833caf76b40c433c7efc0a columnFamilyName family2 2023-03-10 18:56:14,374 INFO [StoreOpener-e86ad0a683833caf76b40c433c7efc0a-1] regionserver.HStore(310): Store=e86ad0a683833caf76b40c433c7efc0a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:56:14,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:14,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:14,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:56:14,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:14,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:56:14,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e86ad0a683833caf76b40c433c7efc0a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68675611, jitterRate=0.023346349596977234}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:56:14,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e86ad0a683833caf76b40c433c7efc0a: 2023-03-10 18:56:14,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a., pid=121, masterSystemTime=1678474574344 2023-03-10 18:56:14,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:14,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:14,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:14,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 21cbab1dd7bfd57fa395887a9422884e, NAME => 'testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e.', STARTKEY => '2', ENDKEY => ''} 2023-03-10 18:56:14,381 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=e86ad0a683833caf76b40c433c7efc0a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:14,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 21cbab1dd7bfd57fa395887a9422884e 2023-03-10 18:56:14,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:56:14,381 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678474574381"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474574381"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474574381"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474574381"}]},"ts":"1678474574381"} 2023-03-10 18:56:14,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 21cbab1dd7bfd57fa395887a9422884e 2023-03-10 18:56:14,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 21cbab1dd7bfd57fa395887a9422884e 2023-03-10 18:56:14,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=118 2023-03-10 18:56:14,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure e86ad0a683833caf76b40c433c7efc0a, server=jenkins-hbase4.apache.org,43181,1678474414959 in 189 msec 2023-03-10 18:56:14,384 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=e86ad0a683833caf76b40c433c7efc0a, ASSIGN in 347 msec 2023-03-10 18:56:14,401 INFO [StoreOpener-21cbab1dd7bfd57fa395887a9422884e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 21cbab1dd7bfd57fa395887a9422884e 2023-03-10 18:56:14,402 DEBUG [StoreOpener-21cbab1dd7bfd57fa395887a9422884e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1 2023-03-10 18:56:14,402 DEBUG [StoreOpener-21cbab1dd7bfd57fa395887a9422884e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1 2023-03-10 18:56:14,402 INFO [StoreOpener-21cbab1dd7bfd57fa395887a9422884e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 21cbab1dd7bfd57fa395887a9422884e columnFamilyName family1 2023-03-10 18:56:14,403 INFO [StoreOpener-21cbab1dd7bfd57fa395887a9422884e-1] regionserver.HStore(310): Store=21cbab1dd7bfd57fa395887a9422884e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:56:14,403 INFO [StoreOpener-21cbab1dd7bfd57fa395887a9422884e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 21cbab1dd7bfd57fa395887a9422884e 2023-03-10 18:56:14,404 DEBUG [StoreOpener-21cbab1dd7bfd57fa395887a9422884e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2 2023-03-10 18:56:14,405 DEBUG [StoreOpener-21cbab1dd7bfd57fa395887a9422884e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2 2023-03-10 18:56:14,405 INFO [StoreOpener-21cbab1dd7bfd57fa395887a9422884e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 21cbab1dd7bfd57fa395887a9422884e columnFamilyName family2 2023-03-10 18:56:14,406 INFO [StoreOpener-21cbab1dd7bfd57fa395887a9422884e-1] regionserver.HStore(310): Store=21cbab1dd7bfd57fa395887a9422884e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:56:14,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e 2023-03-10 18:56:14,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e 2023-03-10 18:56:14,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:56:14,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 21cbab1dd7bfd57fa395887a9422884e 2023-03-10 18:56:14,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:56:14,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 21cbab1dd7bfd57fa395887a9422884e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71001914, jitterRate=0.05801096558570862}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:56:14,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 21cbab1dd7bfd57fa395887a9422884e: 2023-03-10 18:56:14,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e., pid=122, masterSystemTime=1678474574344 2023-03-10 18:56:14,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:14,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:14,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:14,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 40054fa4a1cc8f5fa36fc70cc0b9a27a, NAME => 'testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a.', STARTKEY => '', ENDKEY => '1'} 2023-03-10 18:56:14,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 40054fa4a1cc8f5fa36fc70cc0b9a27a 2023-03-10 18:56:14,413 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=21cbab1dd7bfd57fa395887a9422884e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:14,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-10 18:56:14,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 40054fa4a1cc8f5fa36fc70cc0b9a27a 2023-03-10 18:56:14,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 40054fa4a1cc8f5fa36fc70cc0b9a27a 2023-03-10 18:56:14,413 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474574413"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474574413"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474574413"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474574413"}]},"ts":"1678474574413"} 2023-03-10 18:56:14,415 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-03-10 18:56:14,416 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure 21cbab1dd7bfd57fa395887a9422884e, server=jenkins-hbase4.apache.org,43181,1678474414959 in 220 msec 2023-03-10 18:56:14,416 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=21cbab1dd7bfd57fa395887a9422884e, ASSIGN in 378 msec 2023-03-10 18:56:14,442 INFO [StoreOpener-40054fa4a1cc8f5fa36fc70cc0b9a27a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 40054fa4a1cc8f5fa36fc70cc0b9a27a 2023-03-10 18:56:14,443 DEBUG [StoreOpener-40054fa4a1cc8f5fa36fc70cc0b9a27a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1 2023-03-10 18:56:14,443 DEBUG [StoreOpener-40054fa4a1cc8f5fa36fc70cc0b9a27a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1 2023-03-10 18:56:14,444 INFO [StoreOpener-40054fa4a1cc8f5fa36fc70cc0b9a27a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 40054fa4a1cc8f5fa36fc70cc0b9a27a columnFamilyName family1 2023-03-10 18:56:14,444 INFO [StoreOpener-40054fa4a1cc8f5fa36fc70cc0b9a27a-1] regionserver.HStore(310): Store=40054fa4a1cc8f5fa36fc70cc0b9a27a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:56:14,445 INFO [StoreOpener-40054fa4a1cc8f5fa36fc70cc0b9a27a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 40054fa4a1cc8f5fa36fc70cc0b9a27a 2023-03-10 18:56:14,446 DEBUG [StoreOpener-40054fa4a1cc8f5fa36fc70cc0b9a27a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2 2023-03-10 18:56:14,446 DEBUG [StoreOpener-40054fa4a1cc8f5fa36fc70cc0b9a27a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2 2023-03-10 18:56:14,446 INFO [StoreOpener-40054fa4a1cc8f5fa36fc70cc0b9a27a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 40054fa4a1cc8f5fa36fc70cc0b9a27a columnFamilyName family2 2023-03-10 18:56:14,447 INFO [StoreOpener-40054fa4a1cc8f5fa36fc70cc0b9a27a-1] regionserver.HStore(310): Store=40054fa4a1cc8f5fa36fc70cc0b9a27a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-10 18:56:14,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a 2023-03-10 18:56:14,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a 2023-03-10 18:56:14,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-10 18:56:14,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 40054fa4a1cc8f5fa36fc70cc0b9a27a 2023-03-10 18:56:14,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-10 18:56:14,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 40054fa4a1cc8f5fa36fc70cc0b9a27a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65148989, jitterRate=-0.029204413294792175}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-10 18:56:14,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 40054fa4a1cc8f5fa36fc70cc0b9a27a: 2023-03-10 18:56:14,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a., pid=120, masterSystemTime=1678474574344 2023-03-10 18:56:14,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:14,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:14,453 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=40054fa4a1cc8f5fa36fc70cc0b9a27a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:14,454 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678474574453"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678474574453"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678474574453"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678474574453"}]},"ts":"1678474574453"} 2023-03-10 18:56:14,456 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-03-10 18:56:14,456 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure 40054fa4a1cc8f5fa36fc70cc0b9a27a, server=jenkins-hbase4.apache.org,43181,1678474414959 in 262 msec 2023-03-10 18:56:14,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-03-10 18:56:14,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=40054fa4a1cc8f5fa36fc70cc0b9a27a, ASSIGN in 419 msec 2023-03-10 18:56:14,457 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-10 18:56:14,458 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678474574457"}]},"ts":"1678474574457"} 2023-03-10 18:56:14,459 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-10 18:56:14,461 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-10 18:56:14,462 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2610 sec 2023-03-10 18:56:15,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-10 18:56:15,309 INFO [Listener at localhost/36413] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-10 18:56:15,315 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:15,315 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:15,316 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:15,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:15,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-10 18:56:15,319 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-10 18:56:15,319 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:15,319 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-10 18:56:15,319 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-10 18:56:15,319 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:15,319 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:15,320 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:15,320 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:15,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:15,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:15,320 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:15,320 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:15,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:15,321 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:15,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:15,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:15,321 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-10 18:56:15,321 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-10 18:56:15,321 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-10 18:56:15,321 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:15,321 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-10 18:56:15,322 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:15,322 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:15,322 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:15,322 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:56:15,322 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:15,322 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. started... 2023-03-10 18:56:15,322 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. started... 2023-03-10 18:56:15,322 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. started... 2023-03-10 18:56:15,323 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 21cbab1dd7bfd57fa395887a9422884e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-10 18:56:15,323 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 40054fa4a1cc8f5fa36fc70cc0b9a27a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-10 18:56:15,323 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e86ad0a683833caf76b40c433c7efc0a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-10 18:56:15,369 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310e1e50975f93b4542bd92e15c7c884d25 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310e1e50975f93b4542bd92e15c7c884d25 2023-03-10 18:56:15,766 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031080b401149d5946c2aac1e38cf9883c38 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023031080b401149d5946c2aac1e38cf9883c38 2023-03-10 18:56:15,766 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310f923dd36616d46798e78a47082a8c373 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230310f923dd36616d46798e78a47082a8c373 2023-03-10 18:56:15,783 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family1/06376dc8165f43f19e65e9769481f294 2023-03-10 18:56:15,784 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family1/4806f1985e984153ae07a01e379ba622 2023-03-10 18:56:15,831 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310dd27212ad68b4a44b828549aa6ca701c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230310dd27212ad68b4a44b828549aa6ca701c 2023-03-10 18:56:16,184 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family1/76c6079a0b45465c836c1f0f94810841 2023-03-10 18:56:16,218 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310ab26912759654220a4db513eddd0ea6b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230310ab26912759654220a4db513eddd0ea6b 2023-03-10 18:56:16,222 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303108cbe07e70a834e5caa3d5bd962756106 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303108cbe07e70a834e5caa3d5bd962756106 2023-03-10 18:56:16,229 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family2/b9a1ae73bf794528a32c9ebc0801d3ac 2023-03-10 18:56:16,233 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family1/06376dc8165f43f19e65e9769481f294 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/06376dc8165f43f19e65e9769481f294 2023-03-10 18:56:16,240 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family2/e4e4ba68733a4d3ea419590aa4101595 2023-03-10 18:56:16,244 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family1/4806f1985e984153ae07a01e379ba622 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/4806f1985e984153ae07a01e379ba622 2023-03-10 18:56:16,260 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/06376dc8165f43f19e65e9769481f294, entries=4, sequenceid=5, filesize=5.5 K 2023-03-10 18:56:16,261 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family2/b9a1ae73bf794528a32c9ebc0801d3ac as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/b9a1ae73bf794528a32c9ebc0801d3ac 2023-03-10 18:56:16,265 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/b9a1ae73bf794528a32c9ebc0801d3ac, entries=2, sequenceid=5, filesize=5.1 K 2023-03-10 18:56:16,265 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 40054fa4a1cc8f5fa36fc70cc0b9a27a in 942ms, sequenceid=5, compaction requested=false 2023-03-10 18:56:16,265 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-10 18:56:16,266 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 40054fa4a1cc8f5fa36fc70cc0b9a27a: 2023-03-10 18:56:16,266 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:16,266 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:56:16,271 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/4806f1985e984153ae07a01e379ba622, entries=4, sequenceid=5, filesize=5.5 K 2023-03-10 18:56:16,272 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family2/e4e4ba68733a4d3ea419590aa4101595 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/e4e4ba68733a4d3ea419590aa4101595 2023-03-10 18:56:16,276 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/e4e4ba68733a4d3ea419590aa4101595, entries=2, sequenceid=5, filesize=5.1 K 2023-03-10 18:56:16,277 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 21cbab1dd7bfd57fa395887a9422884e in 953ms, sequenceid=5, compaction requested=false 2023-03-10 18:56:16,277 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 21cbab1dd7bfd57fa395887a9422884e: 2023-03-10 18:56:16,278 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:16,278 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:56:16,626 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family2/b2aada22b2ab498e99f7df0140382148 2023-03-10 18:56:16,630 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family1/76c6079a0b45465c836c1f0f94810841 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/76c6079a0b45465c836c1f0f94810841 2023-03-10 18:56:16,653 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/76c6079a0b45465c836c1f0f94810841, entries=4, sequenceid=5, filesize=5.5 K 2023-03-10 18:56:16,653 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family2/b2aada22b2ab498e99f7df0140382148 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/b2aada22b2ab498e99f7df0140382148 2023-03-10 18:56:16,657 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/b2aada22b2ab498e99f7df0140382148, entries=2, sequenceid=5, filesize=5.1 K 2023-03-10 18:56:16,658 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for e86ad0a683833caf76b40c433c7efc0a in 1335ms, sequenceid=5, compaction requested=false 2023-03-10 18:56:16,658 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e86ad0a683833caf76b40c433c7efc0a: 2023-03-10 18:56:16,658 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:16,658 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:56:16,658 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:56:16,658 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,658 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-10 18:56:16,658 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-10 18:56:16,662 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:16,662 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:16,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:16,662 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:16,662 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:16,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:16,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:16,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:16,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:16,664 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-10 18:56:16,664 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f0e5b7f[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:16,664 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-10 18:56:16,664 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:16,666 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:16,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:16,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:16,666 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-10 18:56:16,666 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-10 18:56:16,666 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:16,667 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,667 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:16,668 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-10 18:56:16,668 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,668 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:16,668 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-10 18:56:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:16,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:16,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:16,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-10 18:56:16,670 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-10 18:56:16,671 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:16,671 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:16,671 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-10 18:56:16,671 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:16,672 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:16,672 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:16,673 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:16,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:16,673 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:16,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:16,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:16,675 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:16,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:16,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,677 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:16,677 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:16,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:16,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:16,677 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,677 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:16,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-10 18:56:16,678 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:16,677 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:16,678 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:16,678 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,678 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:16,678 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:16,678 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:16,678 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:16,678 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:16,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:16,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:16,678 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-10 18:56:16,778 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:16,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:16,780 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:16,780 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:16,781 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:16,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:16,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:16,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-10 18:56:16,788 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-10 18:56:16,788 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:16,788 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-10 18:56:16,788 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-10 18:56:16,788 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:16,788 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:16,796 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:16,796 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:16,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:16,796 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:16,796 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:16,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:16,797 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:16,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:16,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:16,797 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-10 18:56:16,802 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-10 18:56:16,802 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-10 18:56:16,802 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-10 18:56:16,802 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:16,802 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-10 18:56:16,802 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:16,802 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. started... 2023-03-10 18:56:16,802 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 40054fa4a1cc8f5fa36fc70cc0b9a27a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-10 18:56:16,803 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:16,803 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. started... 2023-03-10 18:56:16,803 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 21cbab1dd7bfd57fa395887a9422884e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-10 18:56:16,804 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:56:16,804 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:16,804 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:16,804 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. started... 2023-03-10 18:56:16,804 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e86ad0a683833caf76b40c433c7efc0a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-10 18:56:16,888 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303106c5c7b4bc3fb4f4986a8f880abb49927 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303106c5c7b4bc3fb4f4986a8f880abb49927 2023-03-10 18:56:17,266 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310b2a4a4e5ec8c4200aa8002fd2c52b5d5 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230310b2a4a4e5ec8c4200aa8002fd2c52b5d5 2023-03-10 18:56:17,270 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303100f748108949f4567862ad6d636513671 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303100f748108949f4567862ad6d636513671 2023-03-10 18:56:17,285 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family1/d9704d5f512e4395a134e6042de29689 2023-03-10 18:56:17,304 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family1/cce2694005754dab8b88c8af9675ad59 2023-03-10 18:56:17,335 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303108902418e24d24f56aec04b80a7d1e3db to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303108902418e24d24f56aec04b80a7d1e3db 2023-03-10 18:56:17,339 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023031088fa82626a144aab8f3a4e8662f8de1c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023031088fa82626a144aab8f3a4e8662f8de1c 2023-03-10 18:56:17,349 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family2/a09b769974ba47a8aa4b2a2a7791578f 2023-03-10 18:56:17,354 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family1/cce2694005754dab8b88c8af9675ad59 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/cce2694005754dab8b88c8af9675ad59 2023-03-10 18:56:17,379 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/cce2694005754dab8b88c8af9675ad59, entries=4, sequenceid=9, filesize=5.5 K 2023-03-10 18:56:17,379 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family2/a09b769974ba47a8aa4b2a2a7791578f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/a09b769974ba47a8aa4b2a2a7791578f 2023-03-10 18:56:17,383 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/a09b769974ba47a8aa4b2a2a7791578f, entries=2, sequenceid=9, filesize=5.1 K 2023-03-10 18:56:17,384 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 21cbab1dd7bfd57fa395887a9422884e in 580ms, sequenceid=9, compaction requested=false 2023-03-10 18:56:17,384 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 21cbab1dd7bfd57fa395887a9422884e: 2023-03-10 18:56:17,384 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:17,384 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:56:17,686 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family1/32e8db5d3f5e4c8a9788e3f341c0eace 2023-03-10 18:56:17,747 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family2/380bdf65b08e46068fce7745ac3e2ec3 2023-03-10 18:56:17,750 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family1/d9704d5f512e4395a134e6042de29689 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/d9704d5f512e4395a134e6042de29689 2023-03-10 18:56:17,773 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/d9704d5f512e4395a134e6042de29689, entries=4, sequenceid=9, filesize=5.5 K 2023-03-10 18:56:17,773 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family2/380bdf65b08e46068fce7745ac3e2ec3 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/380bdf65b08e46068fce7745ac3e2ec3 2023-03-10 18:56:17,776 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/380bdf65b08e46068fce7745ac3e2ec3, entries=2, sequenceid=9, filesize=5.1 K 2023-03-10 18:56:17,777 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for e86ad0a683833caf76b40c433c7efc0a in 973ms, sequenceid=9, compaction requested=false 2023-03-10 18:56:17,777 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e86ad0a683833caf76b40c433c7efc0a: 2023-03-10 18:56:17,777 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:17,777 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:56:18,120 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310dac07603f4f641878b89f05e4ddb3f1e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230310dac07603f4f641878b89f05e4ddb3f1e 2023-03-10 18:56:18,525 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family2/021108abd4ce41b091e1bf5d626f1830 2023-03-10 18:56:18,529 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family1/32e8db5d3f5e4c8a9788e3f341c0eace as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/32e8db5d3f5e4c8a9788e3f341c0eace 2023-03-10 18:56:18,553 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/32e8db5d3f5e4c8a9788e3f341c0eace, entries=4, sequenceid=9, filesize=5.5 K 2023-03-10 18:56:18,554 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family2/021108abd4ce41b091e1bf5d626f1830 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/021108abd4ce41b091e1bf5d626f1830 2023-03-10 18:56:18,558 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/021108abd4ce41b091e1bf5d626f1830, entries=2, sequenceid=9, filesize=5.1 K 2023-03-10 18:56:18,559 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 40054fa4a1cc8f5fa36fc70cc0b9a27a in 1757ms, sequenceid=9, compaction requested=false 2023-03-10 18:56:18,559 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 40054fa4a1cc8f5fa36fc70cc0b9a27a: 2023-03-10 18:56:18,559 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:18,559 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:56:18,559 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:56:18,559 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,559 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-10 18:56:18,559 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-10 18:56:18,561 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:18,561 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:18,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:18,561 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:18,561 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:18,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:18,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:18,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:18,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:18,562 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-10 18:56:18,562 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@515d337f[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:18,562 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-10 18:56:18,563 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:18,563 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:18,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:18,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-10 18:56:18,564 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,564 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-10 18:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:18,566 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,566 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-10 18:56:18,566 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:18,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:18,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:18,566 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-10 18:56:18,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:18,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:18,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:18,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:18,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:18,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:18,569 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-10 18:56:18,569 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-10 18:56:18,569 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:18,569 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:18,569 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-10 18:56:18,569 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:18,570 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:18,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:18,570 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:18,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:18,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:18,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:18,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:18,570 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:18,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:18,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:18,571 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:18,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:18,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:18,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:18,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:18,572 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:18,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:18,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,574 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,574 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:18,574 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:18,574 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,574 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:18,574 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:18,574 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-10 18:56:18,574 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:18,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-10 18:56:18,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:18,574 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:18,574 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:18,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:18,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:18,574 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:18,574 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:18,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:18,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:18,574 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:18,675 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:18,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:18,677 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:18,677 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:18,677 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:18,680 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:18,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:18,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-10 18:56:18,682 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-10 18:56:18,682 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:18,682 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-10 18:56:18,682 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-10 18:56:18,682 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:18,683 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:18,684 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,684 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:18,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:18,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:18,684 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:18,684 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:18,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:18,685 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:18,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:18,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:18,685 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-10 18:56:18,685 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-10 18:56:18,685 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-10 18:56:18,685 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-10 18:56:18,685 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:18,685 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-10 18:56:18,686 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:18,686 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:18,686 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. started... 2023-03-10 18:56:18,686 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:56:18,686 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:18,686 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:18,686 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. started... 2023-03-10 18:56:18,686 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. started... 2023-03-10 18:56:18,686 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 40054fa4a1cc8f5fa36fc70cc0b9a27a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-10 18:56:18,686 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 21cbab1dd7bfd57fa395887a9422884e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-10 18:56:18,686 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e86ad0a683833caf76b40c433c7efc0a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-10 18:56:18,742 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310607e0f198a22487b9a21534811ef1ac9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310607e0f198a22487b9a21534811ef1ac9 2023-03-10 18:56:18,755 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303105aafddd303ac41bc830572125ba49e5e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303105aafddd303ac41bc830572125ba49e5e 2023-03-10 18:56:18,758 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310cb2a3d712c154e29b4e87fdd9404f5e2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230310cb2a3d712c154e29b4e87fdd9404f5e2 2023-03-10 18:56:18,766 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family1/3970f98be607446ebe911028f8e7cea2 2023-03-10 18:56:18,775 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family1/5b525583134b44c3b090d756b50758fd 2023-03-10 18:56:18,827 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310cebdba20c6a342c9910cbae6f1c60b57 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230310cebdba20c6a342c9910cbae6f1c60b57 2023-03-10 18:56:19,186 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family1/9e9bc431c16a446cbc21b47a215649ab 2023-03-10 18:56:19,227 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303102c72b915e5514e0d8e5208551be7cc51 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303102c72b915e5514e0d8e5208551be7cc51 2023-03-10 18:56:19,236 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family2/aced74051a5145d2ad9a3eeb12251a11 2023-03-10 18:56:19,237 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family2/be46e3f29f944c48a7523675336cb5a9 2023-03-10 18:56:19,244 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family1/3970f98be607446ebe911028f8e7cea2 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/3970f98be607446ebe911028f8e7cea2 2023-03-10 18:56:19,245 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family1/5b525583134b44c3b090d756b50758fd as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/5b525583134b44c3b090d756b50758fd 2023-03-10 18:56:19,268 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/5b525583134b44c3b090d756b50758fd, entries=4, sequenceid=13, filesize=5.5 K 2023-03-10 18:56:19,269 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family2/be46e3f29f944c48a7523675336cb5a9 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/be46e3f29f944c48a7523675336cb5a9 2023-03-10 18:56:19,273 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/be46e3f29f944c48a7523675336cb5a9, entries=2, sequenceid=13, filesize=5.1 K 2023-03-10 18:56:19,274 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for e86ad0a683833caf76b40c433c7efc0a in 588ms, sequenceid=13, compaction requested=true 2023-03-10 18:56:19,274 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e86ad0a683833caf76b40c433c7efc0a: 2023-03-10 18:56:19,274 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:19,274 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:56:19,277 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/3970f98be607446ebe911028f8e7cea2, entries=4, sequenceid=13, filesize=5.5 K 2023-03-10 18:56:19,277 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family2/aced74051a5145d2ad9a3eeb12251a11 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/aced74051a5145d2ad9a3eeb12251a11 2023-03-10 18:56:19,283 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/aced74051a5145d2ad9a3eeb12251a11, entries=2, sequenceid=13, filesize=5.1 K 2023-03-10 18:56:19,284 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 21cbab1dd7bfd57fa395887a9422884e in 598ms, sequenceid=13, compaction requested=true 2023-03-10 18:56:19,284 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 21cbab1dd7bfd57fa395887a9422884e: 2023-03-10 18:56:19,284 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:19,284 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:56:19,619 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303104c5d4dad23ed458482d9e9448e96eb21 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303104c5d4dad23ed458482d9e9448e96eb21 2023-03-10 18:56:19,892 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-10 18:56:19,952 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:56:19,952 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6da107a7 to 127.0.0.1:56431 2023-03-10 18:56:19,953 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:56:19,953 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:56:19,953 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x650a6872 to 127.0.0.1:56431 2023-03-10 18:56:19,953 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:56:20,023 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family2/d07677068da5495d8dff154ba504b290 2023-03-10 18:56:20,027 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family1/9e9bc431c16a446cbc21b47a215649ab as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/9e9bc431c16a446cbc21b47a215649ab 2023-03-10 18:56:20,049 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/9e9bc431c16a446cbc21b47a215649ab, entries=4, sequenceid=13, filesize=5.5 K 2023-03-10 18:56:20,050 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family2/d07677068da5495d8dff154ba504b290 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/d07677068da5495d8dff154ba504b290 2023-03-10 18:56:20,054 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/d07677068da5495d8dff154ba504b290, entries=2, sequenceid=13, filesize=5.1 K 2023-03-10 18:56:20,054 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 40054fa4a1cc8f5fa36fc70cc0b9a27a in 1368ms, sequenceid=13, compaction requested=true 2023-03-10 18:56:20,054 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 40054fa4a1cc8f5fa36fc70cc0b9a27a: 2023-03-10 18:56:20,054 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:20,054 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:56:20,054 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:56:20,055 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,055 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-10 18:56:20,055 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-10 18:56:20,056 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:20,056 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:20,056 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:20,056 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:20,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:20,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:20,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:20,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:20,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:20,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-10 18:56:20,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@632707d0[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:20,058 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-10 18:56:20,058 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:20,059 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:20,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:20,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:20,060 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-10 18:56:20,060 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-10 18:56:20,060 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:20,060 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,060 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:20,061 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,061 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-10 18:56:20,061 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:20,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:20,061 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-10 18:56:20,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:20,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:20,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:20,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:20,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:20,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:20,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:20,063 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-10 18:56:20,063 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-10 18:56:20,063 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:20,063 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:20,063 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-10 18:56:20,063 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:20,065 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:20,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:20,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:20,065 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:20,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:20,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:20,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:20,065 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:20,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:20,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:20,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:20,066 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:20,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,067 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:20,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:20,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,069 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:20,069 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,069 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:20,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:20,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-10 18:56:20,069 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-10 18:56:20,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:20,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:20,069 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:20,069 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:20,070 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:20,070 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,070 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:20,070 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:20,070 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:20,070 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:20,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:20,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:20,070 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:20,170 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:20,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:20,172 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:20,172 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:20,172 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-10 18:56:20,174 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:20,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:20,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-10 18:56:20,176 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-10 18:56:20,176 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:20,176 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-10 18:56:20,176 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-10 18:56:20,177 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:20,177 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:20,180 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,180 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:20,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:20,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:20,181 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:20,181 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:20,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:20,181 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:20,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:20,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:20,181 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-10 18:56:20,182 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-10 18:56:20,182 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-10 18:56:20,182 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-10 18:56:20,182 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:20,182 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-10 18:56:20,182 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:20,182 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. started... 2023-03-10 18:56:20,182 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 40054fa4a1cc8f5fa36fc70cc0b9a27a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-10 18:56:20,187 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:20,187 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. started... 2023-03-10 18:56:20,187 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:56:20,187 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:20,187 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:20,187 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 21cbab1dd7bfd57fa395887a9422884e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-10 18:56:20,187 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. started... 2023-03-10 18:56:20,187 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e86ad0a683833caf76b40c433c7efc0a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-10 18:56:20,229 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310fddc37bd28d749c09dbf2412ff1e8904 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230310fddc37bd28d749c09dbf2412ff1e8904 2023-03-10 18:56:20,636 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310988efe3d24b4459e90b191a07e6ab1b0 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310988efe3d24b4459e90b191a07e6ab1b0 2023-03-10 18:56:20,636 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310f23280f208fd445097a04a35fca97f81 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230310f23280f208fd445097a04a35fca97f81 2023-03-10 18:56:20,643 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family1/bcbb8ac9056143ebb359608bf048460c 2023-03-10 18:56:20,656 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family1/08c7d0eb62784ac9ae65d46e7e3640f4 2023-03-10 18:56:20,658 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family1/66dc1853ad004a84a69f9e85f8796e0c 2023-03-10 18:56:20,692 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031052cd6b366422468ca903c37f7c85ef88 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023031052cd6b366422468ca903c37f7c85ef88 2023-03-10 18:56:20,696 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303100501e286d5ee48c3848d5ea180e4a82b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303100501e286d5ee48c3848d5ea180e4a82b 2023-03-10 18:56:21,076 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310c68fa2e483e644e89943eda155c4bd11 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230310c68fa2e483e644e89943eda155c4bd11 2023-03-10 18:56:21,081 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family2/03c23cd6819e4295a00a720c1723998e 2023-03-10 18:56:21,084 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family1/bcbb8ac9056143ebb359608bf048460c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/bcbb8ac9056143ebb359608bf048460c 2023-03-10 18:56:21,097 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family2/0f866f299d3a457db524fc3f1973c394 2023-03-10 18:56:21,100 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family2/c007fc4ac27346df9a053def5786ef3f 2023-03-10 18:56:21,104 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family1/66dc1853ad004a84a69f9e85f8796e0c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/66dc1853ad004a84a69f9e85f8796e0c 2023-03-10 18:56:21,107 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family1/08c7d0eb62784ac9ae65d46e7e3640f4 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/08c7d0eb62784ac9ae65d46e7e3640f4 2023-03-10 18:56:21,110 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/bcbb8ac9056143ebb359608bf048460c, entries=4, sequenceid=17, filesize=5.5 K 2023-03-10 18:56:21,111 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family2/03c23cd6819e4295a00a720c1723998e as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/03c23cd6819e4295a00a720c1723998e 2023-03-10 18:56:21,119 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/03c23cd6819e4295a00a720c1723998e, entries=2, sequenceid=17, filesize=5.1 K 2023-03-10 18:56:21,120 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 40054fa4a1cc8f5fa36fc70cc0b9a27a in 938ms, sequenceid=17, compaction requested=true 2023-03-10 18:56:21,120 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 40054fa4a1cc8f5fa36fc70cc0b9a27a: 2023-03-10 18:56:21,120 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:21,120 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:56:21,130 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/66dc1853ad004a84a69f9e85f8796e0c, entries=4, sequenceid=17, filesize=5.5 K 2023-03-10 18:56:21,132 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family2/0f866f299d3a457db524fc3f1973c394 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0f866f299d3a457db524fc3f1973c394 2023-03-10 18:56:21,135 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0f866f299d3a457db524fc3f1973c394, entries=2, sequenceid=17, filesize=5.1 K 2023-03-10 18:56:21,136 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for e86ad0a683833caf76b40c433c7efc0a in 949ms, sequenceid=17, compaction requested=true 2023-03-10 18:56:21,136 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e86ad0a683833caf76b40c433c7efc0a: 2023-03-10 18:56:21,136 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:21,136 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:56:21,141 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/08c7d0eb62784ac9ae65d46e7e3640f4, entries=4, sequenceid=17, filesize=5.5 K 2023-03-10 18:56:21,142 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family2/c007fc4ac27346df9a053def5786ef3f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/c007fc4ac27346df9a053def5786ef3f 2023-03-10 18:56:21,146 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/c007fc4ac27346df9a053def5786ef3f, entries=2, sequenceid=17, filesize=5.1 K 2023-03-10 18:56:21,147 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 21cbab1dd7bfd57fa395887a9422884e in 960ms, sequenceid=17, compaction requested=true 2023-03-10 18:56:21,147 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 21cbab1dd7bfd57fa395887a9422884e: 2023-03-10 18:56:21,147 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:21,147 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:56:21,147 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:56:21,147 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,147 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-10 18:56:21,147 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-10 18:56:21,154 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,154 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:21,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:21,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:21,154 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:21,154 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:21,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:21,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:21,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:21,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:21,156 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-10 18:56:21,156 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33eecf51[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:21,156 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-10 18:56:21,156 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:21,157 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:21,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:21,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:21,158 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-10 18:56:21,158 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-10 18:56:21,158 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:21,158 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,158 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:21,159 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-10 18:56:21,159 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,159 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:21,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:21,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:21,159 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-10 18:56:21,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:21,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:21,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-10 18:56:21,161 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-10 18:56:21,161 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:21,161 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:21,161 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-10 18:56:21,161 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:21,162 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:21,162 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:21,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:21,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:21,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:21,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:21,162 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:21,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:21,163 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:21,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:21,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:21,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:21,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:21,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:21,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:21,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,164 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:21,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:21,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,167 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,167 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:21,167 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:21,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:21,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-10 18:56:21,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:21,167 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:21,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-10 18:56:21,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:21,167 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:21,167 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:21,167 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:21,167 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:21,168 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:21,168 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:21,168 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:21,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:21,168 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:21,268 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:21,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:21,277 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:56:21,284 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:56:21,291 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:56:21,291 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:56:21,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-10 18:56:21,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-10 18:56:21,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-10 18:56:21,392 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-10 18:56:21,392 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:21,392 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-10 18:56:21,392 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-10 18:56:21,392 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:21,392 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:21,393 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,393 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:21,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:21,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:21,394 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:21,394 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-10 18:56:21,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:21,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:21,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-10 18:56:21,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:21,394 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-10 18:56:21,395 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-10 18:56:21,395 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-10 18:56:21,395 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-10 18:56:21,395 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-10 18:56:21,395 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-10 18:56:21,395 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:21,395 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. started... 2023-03-10 18:56:21,395 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 40054fa4a1cc8f5fa36fc70cc0b9a27a 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-10 18:56:21,395 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:21,395 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. started... 2023-03-10 18:56:21,395 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 21cbab1dd7bfd57fa395887a9422884e 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-10 18:56:21,396 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-10 18:56:21,396 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-10 18:56:21,396 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:21,396 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. started... 2023-03-10 18:56:21,396 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e86ad0a683833caf76b40c433c7efc0a 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-10 18:56:21,849 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family1/8298a45e1f274099a2b072263bbc948b 2023-03-10 18:56:21,858 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family1/f9ab6f743e254c04a3d87fd12eae6ed9 2023-03-10 18:56:21,871 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8298a45e1f274099a2b072263bbc948b 2023-03-10 18:56:21,881 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f9ab6f743e254c04a3d87fd12eae6ed9 2023-03-10 18:56:22,245 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family1/0639229725f642bb81b50b0054f39dee 2023-03-10 18:56:22,270 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0639229725f642bb81b50b0054f39dee 2023-03-10 18:56:22,289 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family2/baa8e3a9fcf046f78d1ce4e69c27a259 2023-03-10 18:56:22,292 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family2/cdbaee9df8624c06be0da84211a94a00 2023-03-10 18:56:22,292 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for baa8e3a9fcf046f78d1ce4e69c27a259 2023-03-10 18:56:22,293 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family1/8298a45e1f274099a2b072263bbc948b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/8298a45e1f274099a2b072263bbc948b 2023-03-10 18:56:22,295 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cdbaee9df8624c06be0da84211a94a00 2023-03-10 18:56:22,295 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family1/f9ab6f743e254c04a3d87fd12eae6ed9 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/f9ab6f743e254c04a3d87fd12eae6ed9 2023-03-10 18:56:22,315 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8298a45e1f274099a2b072263bbc948b 2023-03-10 18:56:22,315 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/8298a45e1f274099a2b072263bbc948b, entries=3, sequenceid=23, filesize=5.2 K 2023-03-10 18:56:22,316 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family2/baa8e3a9fcf046f78d1ce4e69c27a259 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/baa8e3a9fcf046f78d1ce4e69c27a259 2023-03-10 18:56:22,317 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f9ab6f743e254c04a3d87fd12eae6ed9 2023-03-10 18:56:22,317 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/f9ab6f743e254c04a3d87fd12eae6ed9, entries=3, sequenceid=23, filesize=5.2 K 2023-03-10 18:56:22,318 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family2/cdbaee9df8624c06be0da84211a94a00 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/cdbaee9df8624c06be0da84211a94a00 2023-03-10 18:56:22,325 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for baa8e3a9fcf046f78d1ce4e69c27a259 2023-03-10 18:56:22,325 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/baa8e3a9fcf046f78d1ce4e69c27a259, entries=1, sequenceid=23, filesize=5.0 K 2023-03-10 18:56:22,325 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cdbaee9df8624c06be0da84211a94a00 2023-03-10 18:56:22,325 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/cdbaee9df8624c06be0da84211a94a00, entries=1, sequenceid=23, filesize=5.0 K 2023-03-10 18:56:22,326 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 40054fa4a1cc8f5fa36fc70cc0b9a27a in 931ms, sequenceid=23, compaction requested=true 2023-03-10 18:56:22,326 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 21cbab1dd7bfd57fa395887a9422884e in 931ms, sequenceid=23, compaction requested=true 2023-03-10 18:56:22,326 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 40054fa4a1cc8f5fa36fc70cc0b9a27a: 2023-03-10 18:56:22,326 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 21cbab1dd7bfd57fa395887a9422884e: 2023-03-10 18:56:22,326 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:22,326 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:22,326 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-10 18:56:22,326 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-10 18:56:22,685 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family2/0e0c22fd7b3b434baa0e843777f786fd 2023-03-10 18:56:22,688 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e0c22fd7b3b434baa0e843777f786fd 2023-03-10 18:56:22,689 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family1/0639229725f642bb81b50b0054f39dee as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/0639229725f642bb81b50b0054f39dee 2023-03-10 18:56:22,712 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0639229725f642bb81b50b0054f39dee 2023-03-10 18:56:22,712 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/0639229725f642bb81b50b0054f39dee, entries=3, sequenceid=23, filesize=5.2 K 2023-03-10 18:56:22,713 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family2/0e0c22fd7b3b434baa0e843777f786fd as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0e0c22fd7b3b434baa0e843777f786fd 2023-03-10 18:56:22,716 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e0c22fd7b3b434baa0e843777f786fd 2023-03-10 18:56:22,717 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0e0c22fd7b3b434baa0e843777f786fd, entries=1, sequenceid=23, filesize=5.0 K 2023-03-10 18:56:22,717 INFO [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for e86ad0a683833caf76b40c433c7efc0a in 1321ms, sequenceid=23, compaction requested=true 2023-03-10 18:56:22,717 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e86ad0a683833caf76b40c433c7efc0a: 2023-03-10 18:56:22,717 DEBUG [rs(jenkins-hbase4.apache.org,43181,1678474414959)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:22,717 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-10 18:56:22,717 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-10 18:56:22,717 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,718 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-10 18:56:22,718 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-10 18:56:22,719 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,719 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:22,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:22,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:22,719 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:22,719 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-10 18:56:22,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:22,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:22,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:22,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:22,721 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43181,1678474414959' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-10 18:56:22,721 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e06aff0[Count = 0] remaining members to acquire global barrier 2023-03-10 18:56:22,721 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-10 18:56:22,721 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:22,722 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:22,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:22,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:22,722 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-10 18:56:22,722 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-10 18:56:22,722 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,43181,1678474414959' in zk 2023-03-10 18:56:22,722 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,722 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-10 18:56:22,725 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-10 18:56:22,725 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,725 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:22,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:22,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:22,725 DEBUG [member: 'jenkins-hbase4.apache.org,43181,1678474414959' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-10 18:56:22,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:22,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:22,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:22,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:22,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:22,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,43181,1678474414959': 2023-03-10 18:56:22,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43181,1678474414959' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-10 18:56:22,728 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-10 18:56:22,728 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-10 18:56:22,728 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-10 18:56:22,728 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-10 18:56:22,728 INFO [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-10 18:56:22,730 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:22,730 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:22,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:22,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:22,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-10 18:56:22,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:22,730 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:22,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-10 18:56:22,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:22,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:22,731 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-10 18:56:22,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:22,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:22,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-10 18:56:22,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:22,732 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-10 18:56:22,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-10 18:56:22,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,734 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,734 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-10 18:56:22,734 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:22,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-10 18:56:22,734 DEBUG [(jenkins-hbase4.apache.org,33867,1678474413725)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-10 18:56:22,734 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-10 18:56:22,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-10 18:56:22,734 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-10 18:56:22,734 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-10 18:56:22,734 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-10 18:56:22,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-10 18:56:22,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-10 18:56:22,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-10 18:56:22,734 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:22,735 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:22,735 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-10 18:56:22,735 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-10 18:56:22,735 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-10 18:56:22,735 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-10 18:56:22,835 DEBUG [Listener at localhost/36413] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-10 18:56:22,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33867] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-10 18:56:22,836 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:56:22,837 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 40054fa4a1cc8f5fa36fc70cc0b9a27a/family1 is initiating major compaction (all files) 2023-03-10 18:56:22,837 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:56:22,837 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:56:22,837 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 40054fa4a1cc8f5fa36fc70cc0b9a27a/family1 in testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:22,837 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/06376dc8165f43f19e65e9769481f294, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/32e8db5d3f5e4c8a9788e3f341c0eace, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/9e9bc431c16a446cbc21b47a215649ab, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/bcbb8ac9056143ebb359608bf048460c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/8298a45e1f274099a2b072263bbc948b] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp, totalSize=27.1 K 2023-03-10 18:56:22,838 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 06376dc8165f43f19e65e9769481f294, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474575315 2023-03-10 18:56:22,838 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 32e8db5d3f5e4c8a9788e3f341c0eace, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474576780 2023-03-10 18:56:22,838 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 9e9bc431c16a446cbc21b47a215649ab, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474578677 2023-03-10 18:56:22,839 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting bcbb8ac9056143ebb359608bf048460c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474580172 2023-03-10 18:56:22,839 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 8298a45e1f274099a2b072263bbc948b, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:56:22,852 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 40054fa4a1cc8f5fa36fc70cc0b9a27a#family1#compaction#197 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:56:23,292 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023031012c384c8d5084b23af62dcbf2618eeb4_del 2023-03-10 18:56:23,292 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031012c384c8d5084b23af62dcbf2618eeb4_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023031012c384c8d5084b23af62dcbf2618eeb4_del 2023-03-10 18:56:24,151 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aef11acd5c0b4bfb9156f4f3fc38279e 2023-03-10 18:56:24,153 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family1/aef11acd5c0b4bfb9156f4f3fc38279e as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/aef11acd5c0b4bfb9156f4f3fc38279e 2023-03-10 18:56:24,191 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aef11acd5c0b4bfb9156f4f3fc38279e 2023-03-10 18:56:24,192 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 40054fa4a1cc8f5fa36fc70cc0b9a27a/family1 of 40054fa4a1cc8f5fa36fc70cc0b9a27a into aef11acd5c0b4bfb9156f4f3fc38279e(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:56:24,192 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 40054fa4a1cc8f5fa36fc70cc0b9a27a: 2023-03-10 18:56:24,193 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:56:24,193 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 40054fa4a1cc8f5fa36fc70cc0b9a27a/family2 is initiating major compaction (all files) 2023-03-10 18:56:24,193 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:56:24,193 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:56:24,194 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 40054fa4a1cc8f5fa36fc70cc0b9a27a/family2 in testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:24,194 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/b9a1ae73bf794528a32c9ebc0801d3ac, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/021108abd4ce41b091e1bf5d626f1830, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/d07677068da5495d8dff154ba504b290, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/03c23cd6819e4295a00a720c1723998e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/baa8e3a9fcf046f78d1ce4e69c27a259] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp, totalSize=25.4 K 2023-03-10 18:56:24,194 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting b9a1ae73bf794528a32c9ebc0801d3ac, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474575315 2023-03-10 18:56:24,195 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 021108abd4ce41b091e1bf5d626f1830, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474576780 2023-03-10 18:56:24,195 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting d07677068da5495d8dff154ba504b290, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474578677 2023-03-10 18:56:24,195 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 03c23cd6819e4295a00a720c1723998e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474580172 2023-03-10 18:56:24,196 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting baa8e3a9fcf046f78d1ce4e69c27a259, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:56:24,203 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 40054fa4a1cc8f5fa36fc70cc0b9a27a#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:56:24,210 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/06376dc8165f43f19e65e9769481f294, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/32e8db5d3f5e4c8a9788e3f341c0eace, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/9e9bc431c16a446cbc21b47a215649ab, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/bcbb8ac9056143ebb359608bf048460c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/8298a45e1f274099a2b072263bbc948b] to archive 2023-03-10 18:56:24,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:24,213 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/06376dc8165f43f19e65e9769481f294 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/06376dc8165f43f19e65e9769481f294 2023-03-10 18:56:24,214 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/32e8db5d3f5e4c8a9788e3f341c0eace to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/32e8db5d3f5e4c8a9788e3f341c0eace 2023-03-10 18:56:24,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/9e9bc431c16a446cbc21b47a215649ab to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/9e9bc431c16a446cbc21b47a215649ab 2023-03-10 18:56:24,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/bcbb8ac9056143ebb359608bf048460c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/bcbb8ac9056143ebb359608bf048460c 2023-03-10 18:56:24,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/8298a45e1f274099a2b072263bbc948b to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/8298a45e1f274099a2b072263bbc948b 2023-03-10 18:56:24,614 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023031001b18bbb29394a05946aee77b9058632_del 2023-03-10 18:56:24,615 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031001b18bbb29394a05946aee77b9058632_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023031001b18bbb29394a05946aee77b9058632_del 2023-03-10 18:56:25,430 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c36a2a25025c47328aa473e167e658ac 2023-03-10 18:56:25,431 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/.tmp/family2/c36a2a25025c47328aa473e167e658ac as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/c36a2a25025c47328aa473e167e658ac 2023-03-10 18:56:25,434 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c36a2a25025c47328aa473e167e658ac 2023-03-10 18:56:25,434 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 40054fa4a1cc8f5fa36fc70cc0b9a27a/family2 of 40054fa4a1cc8f5fa36fc70cc0b9a27a into c36a2a25025c47328aa473e167e658ac(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:56:25,434 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 40054fa4a1cc8f5fa36fc70cc0b9a27a: 2023-03-10 18:56:25,435 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:56:25,435 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 21cbab1dd7bfd57fa395887a9422884e/family1 is initiating major compaction (all files) 2023-03-10 18:56:25,435 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:56:25,435 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:56:25,437 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 21cbab1dd7bfd57fa395887a9422884e/family1 in testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:25,437 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/4806f1985e984153ae07a01e379ba622, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/cce2694005754dab8b88c8af9675ad59, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/3970f98be607446ebe911028f8e7cea2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/08c7d0eb62784ac9ae65d46e7e3640f4, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/f9ab6f743e254c04a3d87fd12eae6ed9] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp, totalSize=27.1 K 2023-03-10 18:56:25,437 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 4806f1985e984153ae07a01e379ba622, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474575316 2023-03-10 18:56:25,437 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting cce2694005754dab8b88c8af9675ad59, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474576781 2023-03-10 18:56:25,438 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 3970f98be607446ebe911028f8e7cea2, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474578677 2023-03-10 18:56:25,438 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 08c7d0eb62784ac9ae65d46e7e3640f4, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474580172 2023-03-10 18:56:25,438 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting f9ab6f743e254c04a3d87fd12eae6ed9, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:56:25,445 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 21cbab1dd7bfd57fa395887a9422884e#family1#compaction#199 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:56:25,510 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/b9a1ae73bf794528a32c9ebc0801d3ac, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/021108abd4ce41b091e1bf5d626f1830, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/d07677068da5495d8dff154ba504b290, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/03c23cd6819e4295a00a720c1723998e, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/baa8e3a9fcf046f78d1ce4e69c27a259] to archive 2023-03-10 18:56:25,511 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:25,512 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/b9a1ae73bf794528a32c9ebc0801d3ac to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/b9a1ae73bf794528a32c9ebc0801d3ac 2023-03-10 18:56:25,513 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/021108abd4ce41b091e1bf5d626f1830 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/021108abd4ce41b091e1bf5d626f1830 2023-03-10 18:56:25,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/d07677068da5495d8dff154ba504b290 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/d07677068da5495d8dff154ba504b290 2023-03-10 18:56:25,515 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/03c23cd6819e4295a00a720c1723998e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/03c23cd6819e4295a00a720c1723998e 2023-03-10 18:56:25,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/baa8e3a9fcf046f78d1ce4e69c27a259 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family2/baa8e3a9fcf046f78d1ce4e69c27a259 2023-03-10 18:56:25,882 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230310a4d93ad5ced24124a416f93b28014aa4_del 2023-03-10 18:56:25,882 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310a4d93ad5ced24124a416f93b28014aa4_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310a4d93ad5ced24124a416f93b28014aa4_del 2023-03-10 18:56:26,744 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 36d52b2f80b643778e9cf57c866c967f 2023-03-10 18:56:26,745 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family1/36d52b2f80b643778e9cf57c866c967f as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/36d52b2f80b643778e9cf57c866c967f 2023-03-10 18:56:26,771 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 36d52b2f80b643778e9cf57c866c967f 2023-03-10 18:56:26,772 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 21cbab1dd7bfd57fa395887a9422884e/family1 of 21cbab1dd7bfd57fa395887a9422884e into 36d52b2f80b643778e9cf57c866c967f(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:56:26,772 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 21cbab1dd7bfd57fa395887a9422884e: 2023-03-10 18:56:26,772 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:56:26,773 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): 21cbab1dd7bfd57fa395887a9422884e/family2 is initiating major compaction (all files) 2023-03-10 18:56:26,773 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:56:26,773 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:56:26,773 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of 21cbab1dd7bfd57fa395887a9422884e/family2 in testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:26,773 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/e4e4ba68733a4d3ea419590aa4101595, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/a09b769974ba47a8aa4b2a2a7791578f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/aced74051a5145d2ad9a3eeb12251a11, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/c007fc4ac27346df9a053def5786ef3f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/cdbaee9df8624c06be0da84211a94a00] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp, totalSize=25.4 K 2023-03-10 18:56:26,774 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting e4e4ba68733a4d3ea419590aa4101595, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474575316 2023-03-10 18:56:26,774 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting a09b769974ba47a8aa4b2a2a7791578f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474576781 2023-03-10 18:56:26,774 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting aced74051a5145d2ad9a3eeb12251a11, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474578677 2023-03-10 18:56:26,775 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting c007fc4ac27346df9a053def5786ef3f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474580172 2023-03-10 18:56:26,775 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting cdbaee9df8624c06be0da84211a94a00, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:56:26,782 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): 21cbab1dd7bfd57fa395887a9422884e#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:56:26,811 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/4806f1985e984153ae07a01e379ba622, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/cce2694005754dab8b88c8af9675ad59, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/3970f98be607446ebe911028f8e7cea2, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/08c7d0eb62784ac9ae65d46e7e3640f4, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/f9ab6f743e254c04a3d87fd12eae6ed9] to archive 2023-03-10 18:56:26,812 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:26,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/4806f1985e984153ae07a01e379ba622 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/4806f1985e984153ae07a01e379ba622 2023-03-10 18:56:26,814 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/cce2694005754dab8b88c8af9675ad59 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/cce2694005754dab8b88c8af9675ad59 2023-03-10 18:56:26,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/3970f98be607446ebe911028f8e7cea2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/3970f98be607446ebe911028f8e7cea2 2023-03-10 18:56:26,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/08c7d0eb62784ac9ae65d46e7e3640f4 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/08c7d0eb62784ac9ae65d46e7e3640f4 2023-03-10 18:56:26,816 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/f9ab6f743e254c04a3d87fd12eae6ed9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/f9ab6f743e254c04a3d87fd12eae6ed9 2023-03-10 18:56:27,189 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230310417f82f8f61f40a2bf75d0e2be27180a_del 2023-03-10 18:56:27,189 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310417f82f8f61f40a2bf75d0e2be27180a_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230310417f82f8f61f40a2bf75d0e2be27180a_del 2023-03-10 18:56:28,003 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 69818e87d69543138f981d50f5674be5 2023-03-10 18:56:28,004 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/.tmp/family2/69818e87d69543138f981d50f5674be5 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/69818e87d69543138f981d50f5674be5 2023-03-10 18:56:28,008 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 69818e87d69543138f981d50f5674be5 2023-03-10 18:56:28,009 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 21cbab1dd7bfd57fa395887a9422884e/family2 of 21cbab1dd7bfd57fa395887a9422884e into 69818e87d69543138f981d50f5674be5(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:56:28,009 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for 21cbab1dd7bfd57fa395887a9422884e: 2023-03-10 18:56:28,009 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:56:28,009 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): e86ad0a683833caf76b40c433c7efc0a/family1 is initiating major compaction (all files) 2023-03-10 18:56:28,009 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:56:28,009 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:56:28,009 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of e86ad0a683833caf76b40c433c7efc0a/family1 in testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:28,009 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/76c6079a0b45465c836c1f0f94810841, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/d9704d5f512e4395a134e6042de29689, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/5b525583134b44c3b090d756b50758fd, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/66dc1853ad004a84a69f9e85f8796e0c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/0639229725f642bb81b50b0054f39dee] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp, totalSize=27.1 K 2023-03-10 18:56:28,010 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 76c6079a0b45465c836c1f0f94810841, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474575315 2023-03-10 18:56:28,010 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting d9704d5f512e4395a134e6042de29689, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474576780 2023-03-10 18:56:28,011 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/e4e4ba68733a4d3ea419590aa4101595, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/a09b769974ba47a8aa4b2a2a7791578f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/aced74051a5145d2ad9a3eeb12251a11, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/c007fc4ac27346df9a053def5786ef3f, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/cdbaee9df8624c06be0da84211a94a00] to archive 2023-03-10 18:56:28,011 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 5b525583134b44c3b090d756b50758fd, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474578677 2023-03-10 18:56:28,011 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:28,011 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 66dc1853ad004a84a69f9e85f8796e0c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474580172 2023-03-10 18:56:28,012 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 0639229725f642bb81b50b0054f39dee, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:56:28,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/e4e4ba68733a4d3ea419590aa4101595 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/e4e4ba68733a4d3ea419590aa4101595 2023-03-10 18:56:28,014 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/a09b769974ba47a8aa4b2a2a7791578f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/a09b769974ba47a8aa4b2a2a7791578f 2023-03-10 18:56:28,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/aced74051a5145d2ad9a3eeb12251a11 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/aced74051a5145d2ad9a3eeb12251a11 2023-03-10 18:56:28,017 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/c007fc4ac27346df9a053def5786ef3f to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/c007fc4ac27346df9a053def5786ef3f 2023-03-10 18:56:28,018 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/cdbaee9df8624c06be0da84211a94a00 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family2/cdbaee9df8624c06be0da84211a94a00 2023-03-10 18:56:28,023 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): e86ad0a683833caf76b40c433c7efc0a#family1#compaction#201 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:56:28,462 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303104ae6ff7ea64048239c3bf39e4d6eae14_del 2023-03-10 18:56:28,462 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303104ae6ff7ea64048239c3bf39e4d6eae14_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303104ae6ff7ea64048239c3bf39e4d6eae14_del 2023-03-10 18:56:29,313 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9955a397ba146bf92e7bc5ab2b19476 2023-03-10 18:56:29,314 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family1/d9955a397ba146bf92e7bc5ab2b19476 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/d9955a397ba146bf92e7bc5ab2b19476 2023-03-10 18:56:29,334 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9955a397ba146bf92e7bc5ab2b19476 2023-03-10 18:56:29,335 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e86ad0a683833caf76b40c433c7efc0a/family1 of e86ad0a683833caf76b40c433c7efc0a into d9955a397ba146bf92e7bc5ab2b19476(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:56:29,335 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for e86ad0a683833caf76b40c433c7efc0a: 2023-03-10 18:56:29,335 DEBUG [Listener at localhost/36413] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-10 18:56:29,336 DEBUG [Listener at localhost/36413] regionserver.HStore(1912): e86ad0a683833caf76b40c433c7efc0a/family2 is initiating major compaction (all files) 2023-03-10 18:56:29,336 INFO [Listener at localhost/36413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-10 18:56:29,336 INFO [Listener at localhost/36413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-10 18:56:29,336 INFO [Listener at localhost/36413] regionserver.HRegion(2259): Starting compaction of e86ad0a683833caf76b40c433c7efc0a/family2 in testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:29,336 INFO [Listener at localhost/36413] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/b2aada22b2ab498e99f7df0140382148, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/380bdf65b08e46068fce7745ac3e2ec3, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/be46e3f29f944c48a7523675336cb5a9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0f866f299d3a457db524fc3f1973c394, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0e0c22fd7b3b434baa0e843777f786fd] into tmpdir=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp, totalSize=25.4 K 2023-03-10 18:56:29,336 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting b2aada22b2ab498e99f7df0140382148, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678474575315 2023-03-10 18:56:29,337 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 380bdf65b08e46068fce7745ac3e2ec3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678474576780 2023-03-10 18:56:29,337 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting be46e3f29f944c48a7523675336cb5a9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678474578677 2023-03-10 18:56:29,337 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 0f866f299d3a457db524fc3f1973c394, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678474580172 2023-03-10 18:56:29,337 DEBUG [Listener at localhost/36413] compactions.Compactor(207): Compacting 0e0c22fd7b3b434baa0e843777f786fd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-10 18:56:29,344 INFO [Listener at localhost/36413] throttle.PressureAwareThroughputController(145): e86ad0a683833caf76b40c433c7efc0a#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-10 18:56:29,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/76c6079a0b45465c836c1f0f94810841, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/d9704d5f512e4395a134e6042de29689, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/5b525583134b44c3b090d756b50758fd, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/66dc1853ad004a84a69f9e85f8796e0c, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/0639229725f642bb81b50b0054f39dee] to archive 2023-03-10 18:56:29,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:29,414 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/76c6079a0b45465c836c1f0f94810841 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/76c6079a0b45465c836c1f0f94810841 2023-03-10 18:56:29,415 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/d9704d5f512e4395a134e6042de29689 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/d9704d5f512e4395a134e6042de29689 2023-03-10 18:56:29,416 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/5b525583134b44c3b090d756b50758fd to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/5b525583134b44c3b090d756b50758fd 2023-03-10 18:56:29,417 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/66dc1853ad004a84a69f9e85f8796e0c to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/66dc1853ad004a84a69f9e85f8796e0c 2023-03-10 18:56:29,418 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/0639229725f642bb81b50b0054f39dee to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/0639229725f642bb81b50b0054f39dee 2023-03-10 18:56:29,750 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230310c309eac699a44876a10e11d31bc03699_del 2023-03-10 18:56:29,750 INFO [Listener at localhost/36413] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230310c309eac699a44876a10e11d31bc03699_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230310c309eac699a44876a10e11d31bc03699_del 2023-03-10 18:56:30,563 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 45f4c74dc0aa4811a016d97399bbe315 2023-03-10 18:56:30,564 DEBUG [Listener at localhost/36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/.tmp/family2/45f4c74dc0aa4811a016d97399bbe315 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/45f4c74dc0aa4811a016d97399bbe315 2023-03-10 18:56:30,568 INFO [Listener at localhost/36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 45f4c74dc0aa4811a016d97399bbe315 2023-03-10 18:56:30,569 INFO [Listener at localhost/36413] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e86ad0a683833caf76b40c433c7efc0a/family2 of e86ad0a683833caf76b40c433c7efc0a into 45f4c74dc0aa4811a016d97399bbe315(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-10 18:56:30,569 DEBUG [Listener at localhost/36413] regionserver.HRegion(2289): Compaction status journal for e86ad0a683833caf76b40c433c7efc0a: 2023-03-10 18:56:30,573 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:56:30,576 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:56:30,581 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:56:30,582 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:56:30,587 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33867] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x1C\xB6\x90\xF7HG\x9D\x1At\xA5\xC6\xC5S\x17)/"\x10\x1F\xFD\xB3\x5C\xFE\x0C\xD9\x9B<1\x92B(sUu*\x10N@/v)\x82\xE6\xDD\x91,\x01\xEF\xAF*\x81\xF62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-10 18:56:30,617 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-10 18:56:30,618 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-10 18:56:30,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/b2aada22b2ab498e99f7df0140382148, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/380bdf65b08e46068fce7745ac3e2ec3, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/be46e3f29f944c48a7523675336cb5a9, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0f866f299d3a457db524fc3f1973c394, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0e0c22fd7b3b434baa0e843777f786fd] to archive 2023-03-10 18:56:30,619 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678475190619 2023-03-10 18:56:30,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:56:30,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:30,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-10 18:56:30,621 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/b2aada22b2ab498e99f7df0140382148 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/b2aada22b2ab498e99f7df0140382148 2023-03-10 18:56:30,624 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/380bdf65b08e46068fce7745ac3e2ec3 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/380bdf65b08e46068fce7745ac3e2ec3 2023-03-10 18:56:30,625 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/be46e3f29f944c48a7523675336cb5a9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/be46e3f29f944c48a7523675336cb5a9 2023-03-10 18:56:30,626 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0f866f299d3a457db524fc3f1973c394 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0f866f299d3a457db524fc3f1973c394 2023-03-10 18:56:30,628 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0e0c22fd7b3b434baa0e843777f786fd to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family2/0e0c22fd7b3b434baa0e843777f786fd 2023-03-10 18:56:31,043 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-10 18:56:31,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-10 18:56:31,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023031012c384c8d5084b23af62dcbf2618eeb4_del 2023-03-10 18:56:31,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303104ae6ff7ea64048239c3bf39e4d6eae14_del 2023-03-10 18:56:31,114 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230310a4d93ad5ced24124a416f93b28014aa4_del 2023-03-10 18:56:31,114 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-10 18:56:31,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] zookeeper.ReadOnlyZKClient(139): Connect 0x59d5c607 to 127.0.0.1:56431 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-10 18:56:31,120 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e96fb63, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-10 18:56:31,120 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230310 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-10 18:56:31,271 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031061249f2f7f304f6c816293bb9576e0e5 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023031061249f2f7f304f6c816293bb9576e0e5 2023-03-10 18:56:31,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230310 table='testMajorCompactionFromAdmin' column='family1' 2023-03-10 18:56:31,282 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-10 18:56:31,285 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56708, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-10 18:56:31,292 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-10 18:56:31,294 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38154, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-10 18:56:31,326 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230310/family1/dd0f798cdbe541e5b0ce30e1e2eb4f91 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-10 18:56:31,327 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230310/family1/dd0f798cdbe541e5b0ce30e1e2eb4f91}] 2023-03-10 18:56:31,332 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230310/family1/dd0f798cdbe541e5b0ce30e1e2eb4f91 for inclusion in e86ad0a683833caf76b40c433c7efc0a/family1 2023-03-10 18:56:31,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-10 18:56:31,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-10 18:56:31,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for e86ad0a683833caf76b40c433c7efc0a: 2023-03-10 18:56:31,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230310/family1/dd0f798cdbe541e5b0ce30e1e2eb4f91 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__vjvt4dqk7gklcri025oucab7p4h0kv299scmco8crrss4ba04e0pothr709l5434/family1/dd0f798cdbe541e5b0ce30e1e2eb4f91 2023-03-10 18:56:31,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__vjvt4dqk7gklcri025oucab7p4h0kv299scmco8crrss4ba04e0pothr709l5434/family1/dd0f798cdbe541e5b0ce30e1e2eb4f91 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/dcaba6c0eae54e5982399592444831d5_SeqId_28_ 2023-03-10 18:56:31,357 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__vjvt4dqk7gklcri025oucab7p4h0kv299scmco8crrss4ba04e0pothr709l5434/family1/dd0f798cdbe541e5b0ce30e1e2eb4f91 into e86ad0a683833caf76b40c433c7efc0a/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/dcaba6c0eae54e5982399592444831d5_SeqId_28_ - updating store file list. 2023-03-10 18:56:31,379 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/dcaba6c0eae54e5982399592444831d5_SeqId_28_ into e86ad0a683833caf76b40c433c7efc0a/family1 2023-03-10 18:56:31,379 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__vjvt4dqk7gklcri025oucab7p4h0kv299scmco8crrss4ba04e0pothr709l5434/family1/dd0f798cdbe541e5b0ce30e1e2eb4f91 into e86ad0a683833caf76b40c433c7efc0a/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/family1/dcaba6c0eae54e5982399592444831d5_SeqId_28_) 2023-03-10 18:56:31,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__vjvt4dqk7gklcri025oucab7p4h0kv299scmco8crrss4ba04e0pothr709l5434/family1/dd0f798cdbe541e5b0ce30e1e2eb4f91 2023-03-10 18:56:31,381 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230310 table='testMajorCompactionFromAdmin' column='family1' 2023-03-10 18:56:31,381 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:31,382 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303105aafddd303ac41bc830572125ba49e5e to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303105aafddd303ac41bc830572125ba49e5e 2023-03-10 18:56:31,383 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230310b2a4a4e5ec8c4200aa8002fd2c52b5d5 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230310b2a4a4e5ec8c4200aa8002fd2c52b5d5 2023-03-10 18:56:31,383 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230310f23280f208fd445097a04a35fca97f81 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230310f23280f208fd445097a04a35fca97f81 2023-03-10 18:56:31,384 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230310f923dd36616d46798e78a47082a8c373 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230310f923dd36616d46798e78a47082a8c373 2023-03-10 18:56:31,384 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-10 18:56:31,384 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230310' 2023-03-10 18:56:31,384 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230310 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-10 18:56:31,384 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230310 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-10 18:56:32,285 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-10 18:56:32,325 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230310ba472276613e4398a25deb8c5468d49a to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310ba472276613e4398a25deb8c5468d49a 2023-03-10 18:56:32,326 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230310 table='testMajorCompactionFromAdmin' column='family1' 2023-03-10 18:56:32,369 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230310/family1/ba519929d9704ca39ec549f04d298ca7 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-10 18:56:32,370 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230310/family1/ba519929d9704ca39ec549f04d298ca7}] 2023-03-10 18:56:32,372 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230310/family1/ba519929d9704ca39ec549f04d298ca7 for inclusion in 21cbab1dd7bfd57fa395887a9422884e/family1 2023-03-10 18:56:32,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-10 18:56:32,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-10 18:56:32,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 21cbab1dd7bfd57fa395887a9422884e: 2023-03-10 18:56:32,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230310/family1/ba519929d9704ca39ec549f04d298ca7 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__thm5cr6a6n3bm0nbl6s4b3aapoh3lvhdhm8g32gg2ba2m34rlropjjrvsjp1hs3o/family1/ba519929d9704ca39ec549f04d298ca7 2023-03-10 18:56:32,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__thm5cr6a6n3bm0nbl6s4b3aapoh3lvhdhm8g32gg2ba2m34rlropjjrvsjp1hs3o/family1/ba519929d9704ca39ec549f04d298ca7 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/e186efb03d3d459a81e3ba563dc2c9e9_SeqId_28_ 2023-03-10 18:56:32,396 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__thm5cr6a6n3bm0nbl6s4b3aapoh3lvhdhm8g32gg2ba2m34rlropjjrvsjp1hs3o/family1/ba519929d9704ca39ec549f04d298ca7 into 21cbab1dd7bfd57fa395887a9422884e/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/e186efb03d3d459a81e3ba563dc2c9e9_SeqId_28_ - updating store file list. 2023-03-10 18:56:32,417 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/e186efb03d3d459a81e3ba563dc2c9e9_SeqId_28_ into 21cbab1dd7bfd57fa395887a9422884e/family1 2023-03-10 18:56:32,418 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__thm5cr6a6n3bm0nbl6s4b3aapoh3lvhdhm8g32gg2ba2m34rlropjjrvsjp1hs3o/family1/ba519929d9704ca39ec549f04d298ca7 into 21cbab1dd7bfd57fa395887a9422884e/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/family1/e186efb03d3d459a81e3ba563dc2c9e9_SeqId_28_) 2023-03-10 18:56:32,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__thm5cr6a6n3bm0nbl6s4b3aapoh3lvhdhm8g32gg2ba2m34rlropjjrvsjp1hs3o/family1/ba519929d9704ca39ec549f04d298ca7 2023-03-10 18:56:32,419 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230310 table='testMajorCompactionFromAdmin' column='family1' 2023-03-10 18:56:32,420 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:32,421 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310607e0f198a22487b9a21534811ef1ac9 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310607e0f198a22487b9a21534811ef1ac9 2023-03-10 18:56:32,422 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303106c5c7b4bc3fb4f4986a8f880abb49927 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303106c5c7b4bc3fb4f4986a8f880abb49927 2023-03-10 18:56:32,422 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310988efe3d24b4459e90b191a07e6ab1b0 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310988efe3d24b4459e90b191a07e6ab1b0 2023-03-10 18:56:32,423 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310e1e50975f93b4542bd92e15c7c884d25 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310e1e50975f93b4542bd92e15c7c884d25 2023-03-10 18:56:32,423 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-10 18:56:32,423 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230310' 2023-03-10 18:56:32,423 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230310 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-10 18:56:32,423 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230310 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-10 18:56:33,358 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230310a3db05fae5344cd38c56ff353d793d52 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230310a3db05fae5344cd38c56ff353d793d52 2023-03-10 18:56:33,359 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230310 table='testMajorCompactionFromAdmin' column='family1' 2023-03-10 18:56:33,399 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230310/family1/7cc356da40b84b8c836895131b9f5519 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-10 18:56:33,400 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a., hostname=jenkins-hbase4.apache.org,43181,1678474414959, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230310/family1/7cc356da40b84b8c836895131b9f5519}] 2023-03-10 18:56:33,402 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(763): Validating hfile at hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230310/family1/7cc356da40b84b8c836895131b9f5519 for inclusion in 40054fa4a1cc8f5fa36fc70cc0b9a27a/family1 2023-03-10 18:56:33,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-10 18:56:33,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(777): Region bounds: first= last=1 2023-03-10 18:56:33,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegion(2446): Flush status journal for 40054fa4a1cc8f5fa36fc70cc0b9a27a: 2023-03-10 18:56:33,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230310/family1/7cc356da40b84b8c836895131b9f5519 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__au7notdbfaap3fljetavamacp7j8laqukt4som9c4o0bjerqi7i89o4g0scb7l1m/family1/7cc356da40b84b8c836895131b9f5519 2023-03-10 18:56:33,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__au7notdbfaap3fljetavamacp7j8laqukt4som9c4o0bjerqi7i89o4g0scb7l1m/family1/7cc356da40b84b8c836895131b9f5519 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/0c2da2a3167145d1a5ebf79f4030e52d_SeqId_28_ 2023-03-10 18:56:33,425 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(849): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__au7notdbfaap3fljetavamacp7j8laqukt4som9c4o0bjerqi7i89o4g0scb7l1m/family1/7cc356da40b84b8c836895131b9f5519 into 40054fa4a1cc8f5fa36fc70cc0b9a27a/family1 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/0c2da2a3167145d1a5ebf79f4030e52d_SeqId_28_ - updating store file list. 2023-03-10 18:56:33,445 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(882): Loaded HFile hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/0c2da2a3167145d1a5ebf79f4030e52d_SeqId_28_ into 40054fa4a1cc8f5fa36fc70cc0b9a27a/family1 2023-03-10 18:56:33,445 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.HStore(855): Successfully loaded hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__au7notdbfaap3fljetavamacp7j8laqukt4som9c4o0bjerqi7i89o4g0scb7l1m/family1/7cc356da40b84b8c836895131b9f5519 into 40054fa4a1cc8f5fa36fc70cc0b9a27a/family1 (new location: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/family1/0c2da2a3167145d1a5ebf79f4030e52d_SeqId_28_) 2023-03-10 18:56:33,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43181] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/staging/jenkins__testMajorCompactionFromAdmin__au7notdbfaap3fljetavamacp7j8laqukt4som9c4o0bjerqi7i89o4g0scb7l1m/family1/7cc356da40b84b8c836895131b9f5519 2023-03-10 18:56:33,447 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230310 table='testMajorCompactionFromAdmin' column='family1' 2023-03-10 18:56:33,448 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:33,449 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303100f748108949f4567862ad6d636513671 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303100f748108949f4567862ad6d636513671 2023-03-10 18:56:33,450 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023031080b401149d5946c2aac1e38cf9883c38 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023031080b401149d5946c2aac1e38cf9883c38 2023-03-10 18:56:33,450 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230310cb2a3d712c154e29b4e87fdd9404f5e2 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230310cb2a3d712c154e29b4e87fdd9404f5e2 2023-03-10 18:56:33,451 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230310fddc37bd28d749c09dbf2412ff1e8904 to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230310fddc37bd28d749c09dbf2412ff1e8904 2023-03-10 18:56:33,451 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-10 18:56:33,451 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230310' 2023-03-10 18:56:33,451 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230310 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-10 18:56:33,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-10 18:56:33,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-10 18:56:33,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023031012c384c8d5084b23af62dcbf2618eeb4_del] 2023-03-10 18:56:33,452 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:33,453 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023031012c384c8d5084b23af62dcbf2618eeb4_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023031012c384c8d5084b23af62dcbf2618eeb4_del 2023-03-10 18:56:33,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-10 18:56:33,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303104ae6ff7ea64048239c3bf39e4d6eae14_del] 2023-03-10 18:56:33,453 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:33,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303104ae6ff7ea64048239c3bf39e4d6eae14_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303104ae6ff7ea64048239c3bf39e4d6eae14_del 2023-03-10 18:56:33,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-10 18:56:33,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310a4d93ad5ced24124a416f93b28014aa4_del] 2023-03-10 18:56:33,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] backup.HFileArchiver(360): Archiving compacted files. 2023-03-10 18:56:33,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310a4d93ad5ced24124a416f93b28014aa4_del to hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230310a4d93ad5ced24124a416f93b28014aa4_del 2023-03-10 18:56:33,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-10 18:56:33,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678474419231] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-10 18:56:33,456 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-10 18:56:33,457 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.8400 sec 2023-03-10 18:56:33,472 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:56:33,473 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:56:33,475 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-10 18:56:33,476 DEBUG [Listener at localhost/36413] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-10 18:56:33,479 INFO [Listener at localhost/36413] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-10 18:56:33,512 INFO [Listener at localhost/36413] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=247 (was 261), OpenFileDescriptor=632 (was 655), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=178 (was 161) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=5243 (was 5414) 2023-03-10 18:56:33,513 DEBUG [Listener at localhost/36413] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x69d78ff9 to 127.0.0.1:56431 2023-03-10 18:56:33,513 DEBUG [Listener at localhost/36413] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:56:33,513 INFO [Listener at localhost/36413] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-10 18:56:33,513 INFO [Listener at localhost/36413] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-10 18:56:33,513 DEBUG [Listener at localhost/36413] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x18f0808a to 127.0.0.1:56431 2023-03-10 18:56:33,513 DEBUG [Listener at localhost/36413] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:56:33,514 DEBUG [Listener at localhost/36413] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-10 18:56:33,514 DEBUG [Listener at localhost/36413] util.JVMClusterUtil(257): Found active master hash=1094977779, stopped=false 2023-03-10 18:56:33,514 INFO [Listener at localhost/36413] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33867,1678474413725 2023-03-10 18:56:33,523 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-10 18:56:33,523 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-10 18:56:33,523 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-10 18:56:33,523 INFO [Listener at localhost/36413] procedure2.ProcedureExecutor(629): Stopping 2023-03-10 18:56:33,523 DEBUG [Listener at localhost/36413] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4d3c7d3c to 127.0.0.1:56431 2023-03-10 18:56:33,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-10 18:56:33,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-10 18:56:33,523 DEBUG [Listener at localhost/36413] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:56:33,524 INFO [Listener at localhost/36413] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43181,1678474414959' ***** 2023-03-10 18:56:33,524 INFO [Listener at localhost/36413] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-10 18:56:33,524 INFO [RS:0;jenkins-hbase4:43181] regionserver.HeapMemoryManager(220): Stopping 2023-03-10 18:56:33,524 INFO [RS:0;jenkins-hbase4:43181] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-10 18:56:33,524 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-10 18:56:33,524 INFO [RS:0;jenkins-hbase4:43181] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-10 18:56:33,524 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for aaa7ef415c2e1954be313179d143a66b 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 75251ceb003f12a816b02e1f9c7cd641 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 7278512a19c794d2cb2655c58a8803cf 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for e46cb331129008feebe80244602e4706 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 6cce5e62535147bca4bf736faaf0f227 2023-03-10 18:56:33,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing aaa7ef415c2e1954be313179d143a66b, disabling compactions & flushes 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 40054fa4a1cc8f5fa36fc70cc0b9a27a 2023-03-10 18:56:33,525 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for b7c0739076470c4809433d5b3c9d4beb 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 21cbab1dd7bfd57fa395887a9422884e 2023-03-10 18:56:33,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 86d4ba0f8509f65063fb59feedda6774 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for a8073efb693951bd51420aa7456ef98c 2023-03-10 18:56:33,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. after waiting 0 ms 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 01b058e52c536aa30e6d0bad142a2e3b 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for a16c84e4f0cb02a5ce455d662efd3ad0 2023-03-10 18:56:33,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 47903cdb8627df6ff46557cb0a143884 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for b416ae27480b11a339c44b606dbd5830 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 6b12c67bd3abfa00d30f0e4b35405679 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 3e4f64f1240b6c6abb2116376732f633 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for d3d50fac089741f712af64c9ad1303fd 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for b1d206b11bc34c2bde324dcef1ee6a89 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 4a6c534dbdc8c65fdb0cb30c4ad6a095 2023-03-10 18:56:33,525 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:33,525 DEBUG [RS:0;jenkins-hbase4:43181] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x23e363a3 to 127.0.0.1:56431 2023-03-10 18:56:33,525 DEBUG [RS:0;jenkins-hbase4:43181] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:56:33,526 INFO [RS:0;jenkins-hbase4:43181] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-10 18:56:33,526 INFO [RS:0;jenkins-hbase4:43181] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-10 18:56:33,526 INFO [RS:0;jenkins-hbase4:43181] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-10 18:56:33,526 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-10 18:56:33,526 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-10 18:56:33,526 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-10 18:56:33,526 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-10 18:56:33,526 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-10 18:56:33,526 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-10 18:56:33,526 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-10 18:56:33,527 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-10 18:56:33,527 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1505): Waiting on 01b058e52c536aa30e6d0bad142a2e3b, 1588230740, 21cbab1dd7bfd57fa395887a9422884e, 3e4f64f1240b6c6abb2116376732f633, 40054fa4a1cc8f5fa36fc70cc0b9a27a, 47903cdb8627df6ff46557cb0a143884, 4a6c534dbdc8c65fdb0cb30c4ad6a095, 6b12c67bd3abfa00d30f0e4b35405679, 6cce5e62535147bca4bf736faaf0f227, 7278512a19c794d2cb2655c58a8803cf, 75251ceb003f12a816b02e1f9c7cd641, 86d4ba0f8509f65063fb59feedda6774, a16c84e4f0cb02a5ce455d662efd3ad0, a8073efb693951bd51420aa7456ef98c, aaa7ef415c2e1954be313179d143a66b, b1d206b11bc34c2bde324dcef1ee6a89, b416ae27480b11a339c44b606dbd5830, b7c0739076470c4809433d5b3c9d4beb, d3d50fac089741f712af64c9ad1303fd, e46cb331129008feebe80244602e4706, e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:33,528 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-10 18:56:33,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithWeeklyPolicy/aaa7ef415c2e1954be313179d143a66b/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-10 18:56:33,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:56:33,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for aaa7ef415c2e1954be313179d143a66b: 2023-03-10 18:56:33,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1678474447946.aaa7ef415c2e1954be313179d143a66b. 2023-03-10 18:56:33,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 75251ceb003f12a816b02e1f9c7cd641, disabling compactions & flushes 2023-03-10 18:56:33,530 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:56:33,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:56:33,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. after waiting 0 ms 2023-03-10 18:56:33,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:56:33,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/75251ceb003f12a816b02e1f9c7cd641/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-10 18:56:33,533 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:56:33,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 75251ceb003f12a816b02e1f9c7cd641: 2023-03-10 18:56:33,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678474491240.75251ceb003f12a816b02e1f9c7cd641. 2023-03-10 18:56:33,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7278512a19c794d2cb2655c58a8803cf, disabling compactions & flushes 2023-03-10 18:56:33,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:56:33,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:56:33,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. after waiting 0 ms 2023-03-10 18:56:33,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:56:33,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/7278512a19c794d2cb2655c58a8803cf/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-10 18:56:33,539 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:56:33,539 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7278512a19c794d2cb2655c58a8803cf: 2023-03-10 18:56:33,539 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1678474513699.7278512a19c794d2cb2655c58a8803cf. 2023-03-10 18:56:33,539 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e46cb331129008feebe80244602e4706, disabling compactions & flushes 2023-03-10 18:56:33,539 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:56:33,539 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:56:33,539 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. after waiting 0 ms 2023-03-10 18:56:33,539 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:56:33,541 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/e46cb331129008feebe80244602e4706/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-10 18:56:33,542 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:56:33,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e46cb331129008feebe80244602e4706: 2023-03-10 18:56:33,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1678474513699.e46cb331129008feebe80244602e4706. 2023-03-10 18:56:33,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6cce5e62535147bca4bf736faaf0f227, disabling compactions & flushes 2023-03-10 18:56:33,542 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:56:33,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:56:33,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. after waiting 0 ms 2023-03-10 18:56:33,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:56:33,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/6cce5e62535147bca4bf736faaf0f227/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-10 18:56:33,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:56:33,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6cce5e62535147bca4bf736faaf0f227: 2023-03-10 18:56:33,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1678474541892.6cce5e62535147bca4bf736faaf0f227. 2023-03-10 18:56:33,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 40054fa4a1cc8f5fa36fc70cc0b9a27a, disabling compactions & flushes 2023-03-10 18:56:33,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:33,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:33,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. after waiting 0 ms 2023-03-10 18:56:33,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:33,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/40054fa4a1cc8f5fa36fc70cc0b9a27a/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-10 18:56:33,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:33,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 40054fa4a1cc8f5fa36fc70cc0b9a27a: 2023-03-10 18:56:33,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1678474573157.40054fa4a1cc8f5fa36fc70cc0b9a27a. 2023-03-10 18:56:33,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b7c0739076470c4809433d5b3c9d4beb, disabling compactions & flushes 2023-03-10 18:56:33,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:56:33,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:56:33,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. after waiting 0 ms 2023-03-10 18:56:33,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:56:33,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/b7c0739076470c4809433d5b3c9d4beb/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-10 18:56:33,552 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:56:33,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b7c0739076470c4809433d5b3c9d4beb: 2023-03-10 18:56:33,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678474476619.b7c0739076470c4809433d5b3c9d4beb. 2023-03-10 18:56:33,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 21cbab1dd7bfd57fa395887a9422884e, disabling compactions & flushes 2023-03-10 18:56:33,552 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:33,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:33,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. after waiting 0 ms 2023-03-10 18:56:33,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:33,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/21cbab1dd7bfd57fa395887a9422884e/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-10 18:56:33,556 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:33,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 21cbab1dd7bfd57fa395887a9422884e: 2023-03-10 18:56:33,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1678474573157.21cbab1dd7bfd57fa395887a9422884e. 2023-03-10 18:56:33,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 86d4ba0f8509f65063fb59feedda6774, disabling compactions & flushes 2023-03-10 18:56:33,556 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:56:33,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:56:33,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. after waiting 0 ms 2023-03-10 18:56:33,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:56:33,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/86d4ba0f8509f65063fb59feedda6774/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-10 18:56:33,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:56:33,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 86d4ba0f8509f65063fb59feedda6774: 2023-03-10 18:56:33,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1678474541892.86d4ba0f8509f65063fb59feedda6774. 2023-03-10 18:56:33,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a8073efb693951bd51420aa7456ef98c, disabling compactions & flushes 2023-03-10 18:56:33,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:56:33,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:56:33,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. after waiting 0 ms 2023-03-10 18:56:33,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:56:33,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a8073efb693951bd51420aa7456ef98c/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-10 18:56:33,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:56:33,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a8073efb693951bd51420aa7456ef98c: 2023-03-10 18:56:33,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678474419409.a8073efb693951bd51420aa7456ef98c. 2023-03-10 18:56:33,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 01b058e52c536aa30e6d0bad142a2e3b, disabling compactions & flushes 2023-03-10 18:56:33,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:56:33,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:56:33,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. after waiting 0 ms 2023-03-10 18:56:33,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:56:33,565 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-10 18:56:33,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithMonthlyPolicy/01b058e52c536aa30e6d0bad142a2e3b/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-10 18:56:33,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:56:33,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 01b058e52c536aa30e6d0bad142a2e3b: 2023-03-10 18:56:33,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1678474434142.01b058e52c536aa30e6d0bad142a2e3b. 2023-03-10 18:56:33,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a16c84e4f0cb02a5ce455d662efd3ad0, disabling compactions & flushes 2023-03-10 18:56:33,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:56:33,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:56:33,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. after waiting 0 ms 2023-03-10 18:56:33,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:56:33,568 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/a16c84e4f0cb02a5ce455d662efd3ad0/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-10 18:56:33,569 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:56:33,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a16c84e4f0cb02a5ce455d662efd3ad0: 2023-03-10 18:56:33,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1678474458769.a16c84e4f0cb02a5ce455d662efd3ad0. 2023-03-10 18:56:33,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 47903cdb8627df6ff46557cb0a143884, disabling compactions & flushes 2023-03-10 18:56:33,569 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:56:33,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:56:33,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. after waiting 0 ms 2023-03-10 18:56:33,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:56:33,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/47903cdb8627df6ff46557cb0a143884/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-10 18:56:33,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:56:33,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 47903cdb8627df6ff46557cb0a143884: 2023-03-10 18:56:33,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1678474458769.47903cdb8627df6ff46557cb0a143884. 2023-03-10 18:56:33,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b416ae27480b11a339c44b606dbd5830, disabling compactions & flushes 2023-03-10 18:56:33,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:56:33,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:56:33,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. after waiting 0 ms 2023-03-10 18:56:33,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:56:33,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionWithWeeklyPolicy/b416ae27480b11a339c44b606dbd5830/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-10 18:56:33,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:56:33,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b416ae27480b11a339c44b606dbd5830: 2023-03-10 18:56:33,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1678474463759.b416ae27480b11a339c44b606dbd5830. 2023-03-10 18:56:33,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6b12c67bd3abfa00d30f0e4b35405679, disabling compactions & flushes 2023-03-10 18:56:33,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:56:33,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:56:33,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. after waiting 0 ms 2023-03-10 18:56:33,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:56:33,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6b12c67bd3abfa00d30f0e4b35405679 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-10 18:56:33,610 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-10 18:56:33,721 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-03-10 18:56:33,721 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-03-10 18:56:33,727 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1505): Waiting on 1588230740, 3e4f64f1240b6c6abb2116376732f633, 4a6c534dbdc8c65fdb0cb30c4ad6a095, 6b12c67bd3abfa00d30f0e4b35405679, b1d206b11bc34c2bde324dcef1ee6a89, d3d50fac089741f712af64c9ad1303fd, e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:33,927 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1505): Waiting on 1588230740, 3e4f64f1240b6c6abb2116376732f633, 4a6c534dbdc8c65fdb0cb30c4ad6a095, 6b12c67bd3abfa00d30f0e4b35405679, b1d206b11bc34c2bde324dcef1ee6a89, d3d50fac089741f712af64c9ad1303fd, e86ad0a683833caf76b40c433c7efc0a 2023-03-10 18:56:33,936 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/.tmp/info/af1c54cc62e84ecaa7ca0088dcb1707b 2023-03-10 18:56:33,940 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af1c54cc62e84ecaa7ca0088dcb1707b 2023-03-10 18:56:33,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679/.tmp/info/88129b6d5e2341bba0f3eab8712a5d93 2023-03-10 18:56:33,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679/.tmp/info/88129b6d5e2341bba0f3eab8712a5d93 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679/info/88129b6d5e2341bba0f3eab8712a5d93 2023-03-10 18:56:33,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679/info/88129b6d5e2341bba0f3eab8712a5d93, entries=3, sequenceid=7, filesize=4.8 K 2023-03-10 18:56:33,991 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 6b12c67bd3abfa00d30f0e4b35405679 in 416ms, sequenceid=7, compaction requested=false 2023-03-10 18:56:33,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/namespace/6b12c67bd3abfa00d30f0e4b35405679/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-10 18:56:33,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:56:33,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6b12c67bd3abfa00d30f0e4b35405679: 2023-03-10 18:56:33,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1678474418109.6b12c67bd3abfa00d30f0e4b35405679. 2023-03-10 18:56:33,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3e4f64f1240b6c6abb2116376732f633, disabling compactions & flushes 2023-03-10 18:56:33,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:56:33,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:56:33,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. after waiting 0 ms 2023-03-10 18:56:33,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:56:34,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/ns/testMinorCompaction/3e4f64f1240b6c6abb2116376732f633/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-10 18:56:34,001 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:56:34,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3e4f64f1240b6c6abb2116376732f633: 2023-03-10 18:56:34,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1678474541892.3e4f64f1240b6c6abb2116376732f633. 2023-03-10 18:56:34,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d3d50fac089741f712af64c9ad1303fd, disabling compactions & flushes 2023-03-10 18:56:34,001 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:34,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:34,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. after waiting 0 ms 2023-03-10 18:56:34,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:34,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMinorCompactionWithMonthlyPolicy/d3d50fac089741f712af64c9ad1303fd/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-10 18:56:34,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:34,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d3d50fac089741f712af64c9ad1303fd: 2023-03-10 18:56:34,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1678474560601.d3d50fac089741f712af64c9ad1303fd. 2023-03-10 18:56:34,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b1d206b11bc34c2bde324dcef1ee6a89, disabling compactions & flushes 2023-03-10 18:56:34,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:56:34,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:56:34,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. after waiting 0 ms 2023-03-10 18:56:34,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:56:34,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testCompactionWithHFileLink/b1d206b11bc34c2bde324dcef1ee6a89/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-10 18:56:34,010 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:56:34,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b1d206b11bc34c2bde324dcef1ee6a89: 2023-03-10 18:56:34,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1678474513699.b1d206b11bc34c2bde324dcef1ee6a89. 2023-03-10 18:56:34,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e86ad0a683833caf76b40c433c7efc0a, disabling compactions & flushes 2023-03-10 18:56:34,010 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:34,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:34,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. after waiting 0 ms 2023-03-10 18:56:34,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:34,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testMajorCompactionFromAdmin/e86ad0a683833caf76b40c433c7efc0a/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-10 18:56:34,013 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:34,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e86ad0a683833caf76b40c433c7efc0a: 2023-03-10 18:56:34,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1678474573157.e86ad0a683833caf76b40c433c7efc0a. 2023-03-10 18:56:34,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4a6c534dbdc8c65fdb0cb30c4ad6a095, disabling compactions & flushes 2023-03-10 18:56:34,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:56:34,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:56:34,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. after waiting 0 ms 2023-03-10 18:56:34,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:56:34,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/default/testScannerOnBulkLoadRefHFiles/4a6c534dbdc8c65fdb0cb30c4ad6a095/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-10 18:56:34,016 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:56:34,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4a6c534dbdc8c65fdb0cb30c4ad6a095: 2023-03-10 18:56:34,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1678474458769.4a6c534dbdc8c65fdb0cb30c4ad6a095. 2023-03-10 18:56:34,128 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-10 18:56:34,328 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-10 18:56:34,345 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/.tmp/rep_barrier/58106e08a1b9415abe30f7f5ec29d19c 2023-03-10 18:56:34,348 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58106e08a1b9415abe30f7f5ec29d19c 2023-03-10 18:56:34,528 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-03-10 18:56:34,528 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-03-10 18:56:34,528 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-10 18:56:34,729 DEBUG [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-10 18:56:34,754 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/.tmp/table/662950aab8fd4ba796e2bf7ee5a4ddc4 2023-03-10 18:56:34,757 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 662950aab8fd4ba796e2bf7ee5a4ddc4 2023-03-10 18:56:34,758 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/.tmp/info/af1c54cc62e84ecaa7ca0088dcb1707b as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/info/af1c54cc62e84ecaa7ca0088dcb1707b 2023-03-10 18:56:34,761 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af1c54cc62e84ecaa7ca0088dcb1707b 2023-03-10 18:56:34,761 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/info/af1c54cc62e84ecaa7ca0088dcb1707b, entries=226, sequenceid=130, filesize=37.9 K 2023-03-10 18:56:34,762 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/.tmp/rep_barrier/58106e08a1b9415abe30f7f5ec29d19c as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/rep_barrier/58106e08a1b9415abe30f7f5ec29d19c 2023-03-10 18:56:34,765 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58106e08a1b9415abe30f7f5ec29d19c 2023-03-10 18:56:34,766 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/rep_barrier/58106e08a1b9415abe30f7f5ec29d19c, entries=1, sequenceid=130, filesize=5.0 K 2023-03-10 18:56:34,766 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/.tmp/table/662950aab8fd4ba796e2bf7ee5a4ddc4 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/table/662950aab8fd4ba796e2bf7ee5a4ddc4 2023-03-10 18:56:34,769 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 662950aab8fd4ba796e2bf7ee5a4ddc4 2023-03-10 18:56:34,769 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/table/662950aab8fd4ba796e2bf7ee5a4ddc4, entries=27, sequenceid=130, filesize=6.6 K 2023-03-10 18:56:34,771 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1245ms, sequenceid=130, compaction requested=false 2023-03-10 18:56:34,775 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-10 18:56:34,775 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-10 18:56:34,775 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-10 18:56:34,775 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-10 18:56:34,775 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-10 18:56:34,805 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-10 18:56:34,929 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43181,1678474414959; all regions closed. 2023-03-10 18:56:34,937 DEBUG [RS:0;jenkins-hbase4:43181] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/oldWALs 2023-03-10 18:56:34,937 INFO [RS:0;jenkins-hbase4:43181] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43181%2C1678474414959.meta:.meta(num 1678474417826) 2023-03-10 18:56:34,949 DEBUG [RS:0;jenkins-hbase4:43181] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/oldWALs 2023-03-10 18:56:34,949 INFO [RS:0;jenkins-hbase4:43181] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43181%2C1678474414959:(num 1678474416877) 2023-03-10 18:56:34,949 DEBUG [RS:0;jenkins-hbase4:43181] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:56:34,950 INFO [RS:0;jenkins-hbase4:43181] regionserver.LeaseManager(133): Closed leases 2023-03-10 18:56:34,950 INFO [RS:0;jenkins-hbase4:43181] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-10 18:56:34,950 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-10 18:56:34,952 INFO [RS:0;jenkins-hbase4:43181] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43181 2023-03-10 18:56:34,960 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43181,1678474414959 2023-03-10 18:56:34,960 ERROR [Listener at localhost/36413-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@79283030 rejected from java.util.concurrent.ThreadPoolExecutor@7d17abcd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-10 18:56:34,960 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-10 18:56:34,960 ERROR [Listener at localhost/36413-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@31f6ef9c rejected from java.util.concurrent.ThreadPoolExecutor@7d17abcd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-10 18:56:34,960 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-10 18:56:34,961 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43181,1678474414959] 2023-03-10 18:56:34,962 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43181,1678474414959; numProcessing=1 2023-03-10 18:56:34,963 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43181,1678474414959 already deleted, retry=false 2023-03-10 18:56:34,963 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43181,1678474414959 expired; onlineServers=0 2023-03-10 18:56:34,963 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33867,1678474413725' ***** 2023-03-10 18:56:34,963 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-10 18:56:34,964 DEBUG [M:0;jenkins-hbase4:33867] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7902c8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-10 18:56:34,964 INFO [M:0;jenkins-hbase4:33867] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33867,1678474413725 2023-03-10 18:56:34,964 INFO [M:0;jenkins-hbase4:33867] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33867,1678474413725; all regions closed. 2023-03-10 18:56:34,964 DEBUG [M:0;jenkins-hbase4:33867] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-10 18:56:34,964 DEBUG [M:0;jenkins-hbase4:33867] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-10 18:56:34,964 DEBUG [M:0;jenkins-hbase4:33867] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-10 18:56:34,964 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-10 18:56:34,964 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678474416556] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678474416556,5,FailOnTimeoutGroup] 2023-03-10 18:56:34,964 INFO [M:0;jenkins-hbase4:33867] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-10 18:56:34,964 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678474416565] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678474416565,5,FailOnTimeoutGroup] 2023-03-10 18:56:34,965 INFO [M:0;jenkins-hbase4:33867] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-10 18:56:34,965 INFO [M:0;jenkins-hbase4:33867] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-03-10 18:56:34,965 DEBUG [M:0;jenkins-hbase4:33867] master.HMaster(1512): Stopping service threads 2023-03-10 18:56:34,965 INFO [M:0;jenkins-hbase4:33867] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-10 18:56:34,966 ERROR [M:0;jenkins-hbase4:33867] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-10 18:56:34,966 INFO [M:0;jenkins-hbase4:33867] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-10 18:56:34,966 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-10 18:56:34,967 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-03-10 18:56:34,968 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-10 18:56:34,968 DEBUG [M:0;jenkins-hbase4:33867] zookeeper.ZKUtil(398): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-10 18:56:34,968 WARN [M:0;jenkins-hbase4:33867] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-10 18:56:34,968 INFO [M:0;jenkins-hbase4:33867] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-10 18:56:34,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-10 18:56:34,968 INFO [M:0;jenkins-hbase4:33867] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-10 18:56:34,968 DEBUG [M:0;jenkins-hbase4:33867] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-10 18:56:34,968 INFO [M:0;jenkins-hbase4:33867] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-10 18:56:34,968 DEBUG [M:0;jenkins-hbase4:33867] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-10 18:56:34,968 DEBUG [M:0;jenkins-hbase4:33867] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-10 18:56:34,968 DEBUG [M:0;jenkins-hbase4:33867] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-10 18:56:34,968 INFO [M:0;jenkins-hbase4:33867] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.48 KB heapSize=598.17 KB 2023-03-10 18:56:34,994 INFO [M:0;jenkins-hbase4:33867] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.48 KB at sequenceid=1005 (bloomFilter=true), to=hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5fb06a5df9b94d94a8d3e3798941cf91 2023-03-10 18:56:34,997 INFO [M:0;jenkins-hbase4:33867] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5fb06a5df9b94d94a8d3e3798941cf91 2023-03-10 18:56:34,998 DEBUG [M:0;jenkins-hbase4:33867] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5fb06a5df9b94d94a8d3e3798941cf91 as hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5fb06a5df9b94d94a8d3e3798941cf91 2023-03-10 18:56:35,001 INFO [M:0;jenkins-hbase4:33867] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5fb06a5df9b94d94a8d3e3798941cf91 2023-03-10 18:56:35,002 INFO [M:0;jenkins-hbase4:33867] regionserver.HStore(1080): Added hdfs://localhost:38247/user/jenkins/test-data/e012cf36-ce7d-02f8-667d-ed68d05511c5/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5fb06a5df9b94d94a8d3e3798941cf91, entries=123, sequenceid=1005, filesize=33.3 K 2023-03-10 18:56:35,002 INFO [M:0;jenkins-hbase4:33867] regionserver.HRegion(2948): Finished flush of dataSize ~512.48 KB/524782, heapSize ~598.16 KB/612512, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 34ms, sequenceid=1005, compaction requested=false 2023-03-10 18:56:35,004 INFO [M:0;jenkins-hbase4:33867] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-10 18:56:35,004 DEBUG [M:0;jenkins-hbase4:33867] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-10 18:56:35,006 INFO [M:0;jenkins-hbase4:33867] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-10 18:56:35,007 INFO [M:0;jenkins-hbase4:33867] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33867 2023-03-10 18:56:35,007 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-10 18:56:35,012 DEBUG [M:0;jenkins-hbase4:33867] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33867,1678474413725 already deleted, retry=false 2023-03-10 18:56:35,061 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-10 18:56:35,061 INFO [RS:0;jenkins-hbase4:43181] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43181,1678474414959; zookeeper connection closed. 2023-03-10 18:56:35,061 ERROR [Listener at localhost/36413-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@49ee6b10 rejected from java.util.concurrent.ThreadPoolExecutor@7d17abcd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-10 18:56:35,062 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): regionserver:43181-0x100cfc6ed280001, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-10 18:56:35,062 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@18812c94] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@18812c94 2023-03-10 18:56:35,062 ERROR [Listener at localhost/36413-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5eb64bd0 rejected from java.util.concurrent.ThreadPoolExecutor@7d17abcd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-10 18:56:35,062 INFO [Listener at localhost/36413] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-10 18:56:35,161 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-10 18:56:35,161 INFO [M:0;jenkins-hbase4:33867] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33867,1678474413725; zookeeper connection closed. 2023-03-10 18:56:35,162 ERROR [Listener at localhost/36413-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@71fa0aa9 rejected from java.util.concurrent.ThreadPoolExecutor@6685073a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-10 18:56:35,162 DEBUG [Listener at localhost/36413-EventThread] zookeeper.ZKWatcher(600): master:33867-0x100cfc6ed280000, quorum=127.0.0.1:56431, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-10 18:56:35,162 ERROR [Listener at localhost/36413-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@12fd08c3 rejected from java.util.concurrent.ThreadPoolExecutor@6685073a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-10 18:56:35,163 WARN [Listener at localhost/36413] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-10 18:56:35,166 INFO [Listener at localhost/36413] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-10 18:56:35,274 WARN [BP-875478649-172.31.14.131-1678474410466 heartbeating to localhost/127.0.0.1:38247] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-10 18:56:35,274 WARN [BP-875478649-172.31.14.131-1678474410466 heartbeating to localhost/127.0.0.1:38247] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-875478649-172.31.14.131-1678474410466 (Datanode Uuid 9b110239-679c-48ee-87dd-e592745d4fb2) service to localhost/127.0.0.1:38247 2023-03-10 18:56:35,276 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/cluster_7b2047bb-341b-b20b-9460-0c7ebaa01809/dfs/data/data1/current/BP-875478649-172.31.14.131-1678474410466] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-10 18:56:35,277 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d46e4e0-737e-5f5a-75bd-1d205b04c06c/cluster_7b2047bb-341b-b20b-9460-0c7ebaa01809/dfs/data/data2/current/BP-875478649-172.31.14.131-1678474410466] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-10 18:56:35,302 INFO [Listener at localhost/36413] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-10 18:56:35,440 INFO [Listener at localhost/36413] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-10 18:56:35,536 INFO [Listener at localhost/36413] hbase.HBaseTestingUtility(1293): Minicluster is down